Jul 07 15:00:32.093103 ip-10-0-138-233 systemd[1]: Starting Kubernetes Kubelet...
Jul 07 15:00:32.527962 ip-10-0-138-233 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 15:00:32.527962 ip-10-0-138-233 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 07 15:00:32.527962 ip-10-0-138-233 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 15:00:32.527962 ip-10-0-138-233 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 07 15:00:32.527962 ip-10-0-138-233 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 15:00:32.529099 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.529016    2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 07 15:00:32.535344 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535315    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 15:00:32.535344 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535336    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 15:00:32.535344 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535340    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 15:00:32.535344 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535344    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 15:00:32.535344 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535347    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 15:00:32.535344 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535350    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 15:00:32.535344 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535353    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 15:00:32.535589 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535358    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 15:00:32.535589 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535362    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 15:00:32.535589 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535365    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 15:00:32.535589 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535368    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 15:00:32.535589 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535372    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 15:00:32.535589 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535374    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 15:00:32.535589 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535377    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 15:00:32.535589 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535405    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 15:00:32.535589 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535409    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 15:00:32.535589 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535412    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 15:00:32.535589 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535414    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 15:00:32.535589 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535417    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 15:00:32.535589 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535420    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 15:00:32.535589 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535423    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 15:00:32.535589 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535426    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 15:00:32.535589 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535429    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 15:00:32.535589 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535433    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 15:00:32.535589 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535436    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 15:00:32.535589 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535439    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 15:00:32.536073 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535442    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 15:00:32.536073 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535444    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 15:00:32.536073 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535447    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 15:00:32.536073 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535451    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 15:00:32.536073 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535454    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 15:00:32.536073 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535456    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 15:00:32.536073 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535459    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 15:00:32.536073 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535462    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 15:00:32.536073 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535464    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 15:00:32.536073 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535467    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 15:00:32.536073 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535469    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 15:00:32.536073 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535471    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 15:00:32.536073 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535474    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 15:00:32.536073 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535476    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 15:00:32.536073 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535479    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 15:00:32.536073 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535482    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 15:00:32.536073 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535485    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 15:00:32.536073 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535487    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 15:00:32.536073 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535490    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 15:00:32.536073 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535493    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 15:00:32.536563 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535495    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 15:00:32.536563 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535498    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 15:00:32.536563 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535501    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 15:00:32.536563 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535503    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 15:00:32.536563 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535505    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 15:00:32.536563 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535507    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 15:00:32.536563 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535510    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 15:00:32.536563 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535512    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 15:00:32.536563 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535515    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 15:00:32.536563 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535517    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 15:00:32.536563 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535519    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 15:00:32.536563 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535522    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 15:00:32.536563 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535525    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 15:00:32.536563 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535527    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 15:00:32.536563 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535529    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 15:00:32.536563 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535532    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 15:00:32.536563 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535535    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 15:00:32.536563 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535538    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 15:00:32.536563 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535541    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 15:00:32.537076 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535545    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 15:00:32.537076 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535548    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 15:00:32.537076 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535551    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 15:00:32.537076 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535554    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 15:00:32.537076 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535556    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 15:00:32.537076 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535559    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 15:00:32.537076 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535561    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 15:00:32.537076 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535563    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 15:00:32.537076 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535566    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 15:00:32.537076 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535568    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 15:00:32.537076 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535570    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 15:00:32.537076 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535573    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 15:00:32.537076 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535575    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 15:00:32.537076 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535578    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 15:00:32.537076 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535580    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 15:00:32.537076 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535583    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 15:00:32.537076 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535585    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 15:00:32.537076 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535588    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 15:00:32.537076 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535590    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 15:00:32.537076 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535592    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 15:00:32.537557 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535595    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 15:00:32.537557 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535597    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 15:00:32.537557 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535600    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 15:00:32.537557 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535602    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 15:00:32.537557 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535605    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 15:00:32.537557 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535608    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 15:00:32.537557 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535610    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 15:00:32.537557 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535612    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 15:00:32.537557 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535615    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 15:00:32.537557 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535617    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 15:00:32.537557 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535620    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 15:00:32.537557 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535623    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 15:00:32.537557 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535625    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 15:00:32.537557 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.535628    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 15:00:32.538256 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538243    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 15:00:32.538287 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538256    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 15:00:32.538287 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538260    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 15:00:32.538287 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538264    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 15:00:32.538287 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538267    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 15:00:32.538287 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538270    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 15:00:32.538287 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538273    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 15:00:32.538287 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538275    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 15:00:32.538287 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538278    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 15:00:32.538287 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538282    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 15:00:32.538287 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538285    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 15:00:32.538287 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538289    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 15:00:32.538287 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538292    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 15:00:32.538587 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538295    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 15:00:32.538587 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538298    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 15:00:32.538587 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538301    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 15:00:32.538587 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538303    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 15:00:32.538587 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538306    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 15:00:32.538587 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538308    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 15:00:32.538587 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538310    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 15:00:32.538587 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538313    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 15:00:32.538587 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538315    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 15:00:32.538587 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538318    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 15:00:32.538587 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538321    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 15:00:32.538587 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538323    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 15:00:32.538587 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538325    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 15:00:32.538587 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538328    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 15:00:32.538587 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538331    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 15:00:32.538587 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538333    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 15:00:32.538587 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538337    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 15:00:32.538587 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538339    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 15:00:32.538587 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538341    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 15:00:32.538587 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538344    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 15:00:32.539091 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538346    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 15:00:32.539091 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538348    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 15:00:32.539091 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538351    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 15:00:32.539091 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538353    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 15:00:32.539091 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538356    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 15:00:32.539091 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538358    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 15:00:32.539091 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538360    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 15:00:32.539091 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538363    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 15:00:32.539091 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538365    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 15:00:32.539091 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538368    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 15:00:32.539091 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538371    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 15:00:32.539091 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538373    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 15:00:32.539091 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538375    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 15:00:32.539091 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538379    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 15:00:32.539091 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538381    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 15:00:32.539091 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538384    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 15:00:32.539091 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538386    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 15:00:32.539091 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538389    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 15:00:32.539091 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538391    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 15:00:32.539091 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538394    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 15:00:32.539658 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538396    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 15:00:32.539658 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538399    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 15:00:32.539658 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538401    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 15:00:32.539658 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538404    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 15:00:32.539658 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538406    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 15:00:32.539658 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538408    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 15:00:32.539658 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538411    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 15:00:32.539658 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538414    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 15:00:32.539658 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538416    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 15:00:32.539658 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538420    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 15:00:32.539658 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538422    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 15:00:32.539658 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538425    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 15:00:32.539658 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538427    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 15:00:32.539658 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538430    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 15:00:32.539658 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538434    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 15:00:32.539658 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538437    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 15:00:32.539658 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538440    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 15:00:32.539658 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538442    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 15:00:32.539658 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538445    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 15:00:32.539658 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538447    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 15:00:32.540174 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538451    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 15:00:32.540174 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538454    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 15:00:32.540174 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538457    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 15:00:32.540174 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538459    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 15:00:32.540174 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538462    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 15:00:32.540174 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538466    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 15:00:32.540174 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538468    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 15:00:32.540174 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538471    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 15:00:32.540174 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538473    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 15:00:32.540174 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538476    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 15:00:32.540174 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538478    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 15:00:32.540174 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538481    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 15:00:32.540174 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538483    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 15:00:32.540174 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538486    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 15:00:32.540174 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538488    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 15:00:32.540174 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538490    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 15:00:32.540174 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538493    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 15:00:32.540174 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538495    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 15:00:32.540174 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538498    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 15:00:32.540627 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538500    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 15:00:32.540627 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538502    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 15:00:32.540627 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538505    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 15:00:32.540627 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538507    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 15:00:32.540627 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538510    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 15:00:32.540627 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538512    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 15:00:32.540627 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538515    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 15:00:32.540627 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538882    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 15:00:32.540627 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538887    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 15:00:32.540627 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538890    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 15:00:32.540627 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538893    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 15:00:32.540627 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538896    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 15:00:32.540627 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538898    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 15:00:32.540627 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538901    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 15:00:32.540627 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538903    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 15:00:32.540627 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538906    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 15:00:32.540627 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538908    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 15:00:32.540627 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538911    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 15:00:32.540627 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538914    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 15:00:32.540627 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538917    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 15:00:32.541145 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538920    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 15:00:32.541145 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538923    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 15:00:32.541145 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538925    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 15:00:32.541145 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538928    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 15:00:32.541145 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538930    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 15:00:32.541145 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538933    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 15:00:32.541145 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538936    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 15:00:32.541145 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538938    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 15:00:32.541145 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538941    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 15:00:32.541145 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538943    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 15:00:32.541145 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538945    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 15:00:32.541145 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538949    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 15:00:32.541145 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538953    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 15:00:32.541145 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538956    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 15:00:32.541145 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538959    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 15:00:32.541145 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538962    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 15:00:32.541145 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538965    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 15:00:32.541145 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538968    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 15:00:32.541145 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538970    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 15:00:32.541607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538973    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 15:00:32.541607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538976    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 15:00:32.541607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538978    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 15:00:32.541607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538981    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 15:00:32.541607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538984    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 15:00:32.541607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538987    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 15:00:32.541607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538989    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 15:00:32.541607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538991    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 15:00:32.541607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538994    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 15:00:32.541607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538996    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 15:00:32.541607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.538998    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 15:00:32.541607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539001    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 15:00:32.541607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539005    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 15:00:32.541607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539009    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 15:00:32.541607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539012    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 15:00:32.541607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539015    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 15:00:32.541607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539017    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 15:00:32.541607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539019    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 15:00:32.541607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539022    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 15:00:32.542126 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539025    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 15:00:32.542126 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539027    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 15:00:32.542126 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539030    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 15:00:32.542126 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539033    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 15:00:32.542126 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539035    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 15:00:32.542126 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539038    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 15:00:32.542126 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539040    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 15:00:32.542126 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539044    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 15:00:32.542126 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539047    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 15:00:32.542126 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539051    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 15:00:32.542126 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539054    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 15:00:32.542126 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539056    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 15:00:32.542126 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539059    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 15:00:32.542126 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539061    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 15:00:32.542126 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539064    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 15:00:32.542126 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539066    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 15:00:32.542126 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539069    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 15:00:32.542126 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539071    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 15:00:32.542126 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539073    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 15:00:32.542576 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539076    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 15:00:32.542576 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539078    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 15:00:32.542576 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539081    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 15:00:32.542576 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539083    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 15:00:32.542576 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539086    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 15:00:32.542576 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539089    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 15:00:32.542576 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539091    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 15:00:32.542576 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539094    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 15:00:32.542576 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539097    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 15:00:32.542576 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539099    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 15:00:32.542576 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539102    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 15:00:32.542576 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539105    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 15:00:32.542576 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539108    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 15:00:32.542576 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539110    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 15:00:32.542576 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539113    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 15:00:32.542576 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539115    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 15:00:32.542576 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539117    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 15:00:32.542576 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539120    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 15:00:32.542576 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539122    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 15:00:32.543055 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539125    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 15:00:32.543055 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539127    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 15:00:32.543055 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539129    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 15:00:32.543055 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539132    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 15:00:32.543055 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539134    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 15:00:32.543055 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539137    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 15:00:32.543055 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539139    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 15:00:32.543055 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539142    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 15:00:32.543055 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539144    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 15:00:32.543055 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539146    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 15:00:32.543055 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539211    2567 flags.go:64] FLAG: --address="0.0.0.0"
Jul 07 15:00:32.543055 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539218    2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 07 15:00:32.543055 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539224    2567 flags.go:64] FLAG: --anonymous-auth="true"
Jul 07 15:00:32.543055 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539228    2567 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 07 15:00:32.543055 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539232    2567 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 07 15:00:32.543055 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539235    2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 07 15:00:32.543055 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539239    2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 07 15:00:32.543055 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539244    2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 07 15:00:32.543055 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539247    2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 07 15:00:32.543055 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539250    2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 07 15:00:32.543055 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539254    2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 07 15:00:32.543055 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539257    2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 07 15:00:32.543597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539260    2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 07 15:00:32.543597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539262    2567 flags.go:64] FLAG: --cgroup-root=""
Jul 07 15:00:32.543597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539265    2567 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 07 15:00:32.543597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539268    2567 flags.go:64] FLAG: --client-ca-file=""
Jul 07 15:00:32.543597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539271    2567 flags.go:64] FLAG: --cloud-provider="external"
Jul 07 15:00:32.543597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539273    2567 flags.go:64] FLAG: --cluster-dns="[]"
Jul 07 15:00:32.543597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539277    2567 flags.go:64] FLAG: --cluster-domain=""
Jul 07 15:00:32.543597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539280    2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 07 15:00:32.543597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539283    2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 07 15:00:32.543597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539286    2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 07 15:00:32.543597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539289    2567 flags.go:64] FLAG: --container-log-max-files="5"
Jul 07 15:00:32.543597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539293    2567 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 07 15:00:32.543597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539295    2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 07 15:00:32.543597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539298    2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 07 15:00:32.543597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539301    2567 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 07 15:00:32.543597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539304    2567 flags.go:64] FLAG: --contention-profiling="false"
Jul 07 15:00:32.543597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539308    2567 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 07 15:00:32.543597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539311    2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 07 15:00:32.543597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539314    2567 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 07 15:00:32.543597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539319    2567 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 07 15:00:32.543597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539323    2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 07 15:00:32.543597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539326    2567 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 07 15:00:32.543597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539329    2567 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 07 15:00:32.543597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539332    2567 flags.go:64] FLAG: --enable-load-reader="false"
Jul 07 15:00:32.543597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539335    2567 flags.go:64] FLAG: --enable-server="true"
Jul 07 15:00:32.544206 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539338    2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 07 15:00:32.544206 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539342    2567 flags.go:64] FLAG: --event-burst="100"
Jul 07 15:00:32.544206 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539345    2567 flags.go:64] FLAG: --event-qps="50"
Jul 07 15:00:32.544206 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539348    2567 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 07 15:00:32.544206 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539351    2567 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 07 15:00:32.544206 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539354    2567 flags.go:64] FLAG: --eviction-hard=""
Jul 07 15:00:32.544206 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539357    2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 07 15:00:32.544206 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539360    2567 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 07 15:00:32.544206 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539363    2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 07 15:00:32.544206 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539365    2567 flags.go:64] FLAG: --eviction-soft=""
Jul 07 15:00:32.544206 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539368    2567 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 07 15:00:32.544206 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539371    2567 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 07 15:00:32.544206 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539374    2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 07 15:00:32.544206 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539376    2567 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 07 15:00:32.544206 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539379    2567 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 07 15:00:32.544206 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539382    2567 flags.go:64] FLAG: --fail-swap-on="true"
Jul 07 15:00:32.544206 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539384    2567 flags.go:64] FLAG: --feature-gates=""
Jul 07 15:00:32.544206 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539388    2567 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 07 15:00:32.544206 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539391    2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 07 15:00:32.544206 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539394    2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 07 15:00:32.544206 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539397    2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 07 15:00:32.544206 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539399    2567 flags.go:64] FLAG: --healthz-port="10248"
Jul 07 15:00:32.544206 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539402    2567 flags.go:64] FLAG: --help="false"
Jul 07 15:00:32.544206 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539405    2567 flags.go:64] FLAG: --hostname-override="ip-10-0-138-233.ec2.internal"
Jul 07 15:00:32.544206 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539409    2567 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 07 15:00:32.544864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539412    2567 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 07 15:00:32.544864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539417    2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 07 15:00:32.544864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539420    2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 07 15:00:32.544864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539423    2567 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 07 15:00:32.544864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539426    2567 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 07 15:00:32.544864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539428    2567 flags.go:64] FLAG: --image-service-endpoint=""
Jul 07 15:00:32.544864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539431    2567 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 07 15:00:32.544864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539433    2567 flags.go:64] FLAG: --kube-api-burst="100"
Jul 07 15:00:32.544864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539436    2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 07 15:00:32.544864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539439    2567 flags.go:64] FLAG: --kube-api-qps="50"
Jul 07 15:00:32.544864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539442    2567 flags.go:64] FLAG: --kube-reserved=""
Jul 07 15:00:32.544864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539445    2567 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 07 15:00:32.544864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539448    2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 07 15:00:32.544864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539451    2567 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 07 15:00:32.544864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539453    2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 07 15:00:32.544864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539456    2567 flags.go:64] FLAG: --lock-file=""
Jul 07 15:00:32.544864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539459    2567 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 07 15:00:32.544864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539462    2567 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 07 15:00:32.544864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539464    2567 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 07 15:00:32.544864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539469    2567 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 07 15:00:32.544864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539473    2567 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 07 15:00:32.544864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539475    2567 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 07 15:00:32.544864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539478    2567 flags.go:64] FLAG: --logging-format="text"
Jul 07 15:00:32.545459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539481    2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 07 15:00:32.545459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539484    2567 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 07 15:00:32.545459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539487    2567 flags.go:64] FLAG: --manifest-url=""
Jul 07 15:00:32.545459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539489    2567 flags.go:64] FLAG: --manifest-url-header=""
Jul 07 15:00:32.545459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539493    2567 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 07 15:00:32.545459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539496    2567 flags.go:64] FLAG: --max-open-files="1000000"
Jul 07 15:00:32.545459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539501    2567 flags.go:64] FLAG: --max-pods="110"
Jul 07 15:00:32.545459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539503    2567 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 07 15:00:32.545459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539506    2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 07 15:00:32.545459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539509    2567 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 07 15:00:32.545459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539514    2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 07 15:00:32.545459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539516    2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 07 15:00:32.545459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539519    2567 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 07 15:00:32.545459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539522    2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 07 15:00:32.545459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539529    2567 flags.go:64] FLAG: --node-status-max-images="50"
Jul 07 15:00:32.545459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539532    2567 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 07 15:00:32.545459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539535    2567 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 07 15:00:32.545459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539537    2567 flags.go:64] FLAG: --pod-cidr=""
Jul 07 15:00:32.545459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539540    2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 07 15:00:32.545459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539545    2567 flags.go:64] FLAG: --pod-manifest-path=""
Jul 07 15:00:32.545459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539548    2567 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 07 15:00:32.545459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539551    2567 flags.go:64] FLAG: --pods-per-core="0"
Jul 07 15:00:32.545459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539553    2567 flags.go:64] FLAG: --port="10250"
Jul 07 15:00:32.545459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539556    2567 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 07 15:00:32.546123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539559    2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0432ab3ba5386588b"
Jul 07 15:00:32.546123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539562    2567 flags.go:64] FLAG: --qos-reserved=""
Jul 07 15:00:32.546123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539564    2567 flags.go:64] FLAG: --read-only-port="10255"
Jul 07 15:00:32.546123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539567    2567 flags.go:64] FLAG: --register-node="true"
Jul 07 15:00:32.546123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539570    2567 flags.go:64] FLAG: --register-with-taints=""
Jul 07 15:00:32.546123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539573    2567 flags.go:64] FLAG: --registry-burst="10"
Jul 07 15:00:32.546123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539576    2567 flags.go:64] FLAG: --registry-qps="5"
Jul 07 15:00:32.546123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539578    2567 flags.go:64] FLAG: --reserved-cpus=""
Jul 07 15:00:32.546123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539581    2567 flags.go:64] FLAG: --reserved-memory=""
Jul 07 15:00:32.546123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539585    2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 07 15:00:32.546123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539588    2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 07 15:00:32.546123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539590    2567 flags.go:64] FLAG: --rotate-certificates="false"
Jul 07 15:00:32.546123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539593    2567 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 07 15:00:32.546123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539596    2567 flags.go:64] FLAG: --runonce="false"
Jul 07 15:00:32.546123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539598    2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 07 15:00:32.546123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539601    2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 07 15:00:32.546123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539604    2567 flags.go:64] FLAG: --seccomp-default="false"
Jul 07 15:00:32.546123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539607    2567 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 07 15:00:32.546123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539610    2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 07 15:00:32.546123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539614    2567 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 07 15:00:32.546123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539618    2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 07 15:00:32.546123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539620    2567 flags.go:64] FLAG: --storage-driver-password="root"
Jul 07 15:00:32.546123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539623    2567 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 07 15:00:32.546123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539626    2567 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 07 15:00:32.546123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539629    2567 flags.go:64] FLAG: --storage-driver-user="root"
Jul 07 15:00:32.546721 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539632    2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 07 15:00:32.546721 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539635    2567 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 07 15:00:32.546721 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539637    2567 flags.go:64] FLAG: --system-cgroups=""
Jul 07 15:00:32.546721 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539640    2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 07 15:00:32.546721 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539645    2567 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 07 15:00:32.546721 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539648    2567 flags.go:64] FLAG: --tls-cert-file=""
Jul 07 15:00:32.546721 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539650    2567 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 07 15:00:32.546721 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539654    2567 flags.go:64] FLAG: --tls-min-version=""
Jul 07 15:00:32.546721 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539657    2567 flags.go:64] FLAG: --tls-private-key-file=""
Jul 07 15:00:32.546721 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539660    2567 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 07 15:00:32.546721 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539663    2567 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 07 15:00:32.546721 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539666    2567 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 07 15:00:32.546721 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539668    2567 flags.go:64] FLAG: --v="2"
Jul 07 15:00:32.546721 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539672    2567 flags.go:64] FLAG: --version="false"
Jul 07 15:00:32.546721 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539676    2567 flags.go:64] FLAG: --vmodule=""
Jul 07 15:00:32.546721 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539679    2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 07 15:00:32.546721 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.539682    2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 07 15:00:32.546721 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539810    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 15:00:32.546721 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539814    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 15:00:32.546721 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539816    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 15:00:32.546721 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539819    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 15:00:32.546721 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539821    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 15:00:32.546721 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539825    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 15:00:32.547297 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539841    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 15:00:32.547297 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539846    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 15:00:32.547297 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539849    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 15:00:32.547297 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539858    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 15:00:32.547297 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539861    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 15:00:32.547297 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539863    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 15:00:32.547297 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539866    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 15:00:32.547297 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539869    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 15:00:32.547297 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539871    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 15:00:32.547297 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539874    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 15:00:32.547297 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539876    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 15:00:32.547297 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539878    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 15:00:32.547297 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539881    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 15:00:32.547297 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539883    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 15:00:32.547297 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539886    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 15:00:32.547297 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539888    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 15:00:32.547297 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539891    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 15:00:32.547297 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539893    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 15:00:32.547297 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539896    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 15:00:32.547297 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539898    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 15:00:32.547918 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539901    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 15:00:32.547918 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539903    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 15:00:32.547918 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539906    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 15:00:32.547918 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539908    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 15:00:32.547918 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539911    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 15:00:32.547918 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539913    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 15:00:32.547918 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539916    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 15:00:32.547918 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539918    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 15:00:32.547918 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539920    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 15:00:32.547918 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539923    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 15:00:32.547918 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539925    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 15:00:32.547918 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539928    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 15:00:32.547918 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539930    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 15:00:32.547918 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539934    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 15:00:32.547918 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539937    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 15:00:32.547918 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539942    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 15:00:32.547918 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539954    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 15:00:32.547918 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539958    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 15:00:32.547918 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539962    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 15:00:32.547918 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539965    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 15:00:32.548632 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539968    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 15:00:32.548632 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539970    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 15:00:32.548632 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539973    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 15:00:32.548632 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539976    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 15:00:32.548632 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539979    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 15:00:32.548632 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539981    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 15:00:32.548632 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539984    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 15:00:32.548632 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539986    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 15:00:32.548632 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539989    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 15:00:32.548632 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539991    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 15:00:32.548632 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539994    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 15:00:32.548632 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539996    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 15:00:32.548632 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.539999    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 15:00:32.548632 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540001    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 15:00:32.548632 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540003    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 15:00:32.548632 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540006    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 15:00:32.548632 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540010    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 15:00:32.548632 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540013    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 15:00:32.548632 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540015    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 15:00:32.549134 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540018    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 15:00:32.549134 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540020    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 15:00:32.549134 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540023    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 15:00:32.549134 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540026    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 15:00:32.549134 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540029    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 15:00:32.549134 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540032    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 15:00:32.549134 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540036    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 15:00:32.549134 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540047    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 15:00:32.549134 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540051    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 15:00:32.549134 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540054    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 15:00:32.549134 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540062    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 15:00:32.549134 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540065    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 15:00:32.549134 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540068    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 15:00:32.549134 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540070    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 15:00:32.549134 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540072    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 15:00:32.549134 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540075    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 15:00:32.549134 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540077    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 15:00:32.549134 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540080    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 15:00:32.549134 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540082    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 15:00:32.549607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540085    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 15:00:32.549607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540087    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 15:00:32.549607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540089    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 15:00:32.549607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540092    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 15:00:32.549607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540094    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 15:00:32.549607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540096    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 15:00:32.549607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540099    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 15:00:32.549607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540101    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 15:00:32.549607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540104    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 15:00:32.549607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540106    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 15:00:32.549607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540108    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 15:00:32.549607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540111    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 15:00:32.549607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540113    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 15:00:32.549607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540116    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 15:00:32.549607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.540118    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 15:00:32.549987 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.540868    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 15:00:32.549987 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.549907    2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 07 15:00:32.549987 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.549924    2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 07 15:00:32.549987 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.549944    2567 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 07 15:00:32.549987 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.549949    2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 07 15:00:32.550121 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550018    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 15:00:32.550121 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550024    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 15:00:32.550121 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550029    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 15:00:32.550121 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550033    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 15:00:32.550121 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550036    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 15:00:32.550121 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550039    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 15:00:32.550121 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550042    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 15:00:32.550121 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550045    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 15:00:32.550121 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550047    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 15:00:32.550121 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550050    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 15:00:32.550121 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550053    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 15:00:32.550121 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550055    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 15:00:32.550121 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550058    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 15:00:32.550121 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550060    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 15:00:32.550121 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550063    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 15:00:32.550121 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550065    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 15:00:32.550121 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550067    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 15:00:32.550121 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550070    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 15:00:32.550121 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550072    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 15:00:32.550583 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550075    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 15:00:32.550583 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550077    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 15:00:32.550583 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550080    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 15:00:32.550583 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550082    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 15:00:32.550583 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550085    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 15:00:32.550583 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550087    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 15:00:32.550583 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550089    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 15:00:32.550583 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550092    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 15:00:32.550583 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550094    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 15:00:32.550583 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550097    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 15:00:32.550583 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550099    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 15:00:32.550583 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550102    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 15:00:32.550583 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550104    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 15:00:32.550583 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550107    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 15:00:32.550583 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550110    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 15:00:32.550583 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550113    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 15:00:32.550583 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550115    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 15:00:32.550583 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550118    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 15:00:32.550583 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550120    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 15:00:32.550583 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550122    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 15:00:32.551140 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550125    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 15:00:32.551140 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550127    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 15:00:32.551140 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550130    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 15:00:32.551140 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550133    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 15:00:32.551140 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550136    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 15:00:32.551140 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550138    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 15:00:32.551140 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550140    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 15:00:32.551140 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550143    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 15:00:32.551140 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550146    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 15:00:32.551140 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550148    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 15:00:32.551140 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550151    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 15:00:32.551140 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550153    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 15:00:32.551140 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550155    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 15:00:32.551140 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550159    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 15:00:32.551140 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550163    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 15:00:32.551140 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550166    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 15:00:32.551140 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550169    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 15:00:32.551140 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550171    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 15:00:32.551140 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550174    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 15:00:32.551642 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550176    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 15:00:32.551642 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550178    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 15:00:32.551642 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550181    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 15:00:32.551642 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550183    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 15:00:32.551642 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550186    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 15:00:32.551642 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550188    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 15:00:32.551642 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550191    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 15:00:32.551642 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550194    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 15:00:32.551642 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550197    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 15:00:32.551642 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550199    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 15:00:32.551642 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550202    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 15:00:32.551642 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550204    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 15:00:32.551642 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550207    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 15:00:32.551642 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550210    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 15:00:32.551642 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550212    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 15:00:32.551642 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550215    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 15:00:32.551642 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550217    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 15:00:32.551642 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550219    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 15:00:32.551642 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550222    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 15:00:32.551642 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550224    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 15:00:32.552146 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550227    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 15:00:32.552146 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550230    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 15:00:32.552146 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550232    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 15:00:32.552146 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550235    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 15:00:32.552146 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550237    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 15:00:32.552146 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550240    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 15:00:32.552146 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550242    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 15:00:32.552146 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550245    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 15:00:32.552146 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550247    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 15:00:32.552146 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550249    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 15:00:32.552146 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550251    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 15:00:32.552146 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550254    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 15:00:32.552146 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550257    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 15:00:32.552146 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550259    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 15:00:32.552146 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550262    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 15:00:32.552146 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550264    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 15:00:32.552146 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550267    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 15:00:32.552146 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550270    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 15:00:32.552146 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550272    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 15:00:32.552146 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550274    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 15:00:32.552635 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550277    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 15:00:32.552635 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.550282    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 15:00:32.552635 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550412    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 15:00:32.552635 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550416    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 15:00:32.552635 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550420    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 15:00:32.552635 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550422    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 15:00:32.552635 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550425    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 15:00:32.552635 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550428    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 15:00:32.552635 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550430    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 15:00:32.552635 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550433    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 15:00:32.552635 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550436    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 15:00:32.552635 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550440    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 15:00:32.552635 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550443    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 15:00:32.552635 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550445    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 15:00:32.552635 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550448    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 15:00:32.553031 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550451    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 15:00:32.553031 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550453    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 15:00:32.553031 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550456    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 15:00:32.553031 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550458    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 15:00:32.553031 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550460    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 15:00:32.553031 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550463    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 15:00:32.553031 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550465    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 15:00:32.553031 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550468    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 15:00:32.553031 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550471    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 15:00:32.553031 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550473    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 15:00:32.553031 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550476    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 15:00:32.553031 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550478    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 15:00:32.553031 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550481    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 15:00:32.553031 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550483    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 15:00:32.553031 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550486    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 15:00:32.553031 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550488    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 15:00:32.553031 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550490    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 15:00:32.553031 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550493    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 15:00:32.553031 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550495    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 15:00:32.553485 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550498    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 15:00:32.553485 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550500    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 15:00:32.553485 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550502    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 15:00:32.553485 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550505    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 15:00:32.553485 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550507    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 15:00:32.553485 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550509    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 15:00:32.553485 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550512    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 15:00:32.553485 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550514    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 15:00:32.553485 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550516    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 15:00:32.553485 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550519    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 15:00:32.553485 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550521    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 15:00:32.553485 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550523    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 15:00:32.553485 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550526    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 15:00:32.553485 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550529    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 15:00:32.553485 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550532    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 15:00:32.553485 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550534    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 15:00:32.553485 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550536    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 15:00:32.553485 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550539    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 15:00:32.553485 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550541    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 15:00:32.553968 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550545    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 15:00:32.553968 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550548    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 15:00:32.553968 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550551    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 15:00:32.553968 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550553    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 15:00:32.553968 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550556    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 15:00:32.553968 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550559    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 15:00:32.553968 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550562    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 15:00:32.553968 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550564    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 15:00:32.553968 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550567    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 15:00:32.553968 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550570    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 15:00:32.553968 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550572    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 15:00:32.553968 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550575    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 15:00:32.553968 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550577    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 15:00:32.553968 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550579    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 15:00:32.553968 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550582    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 15:00:32.553968 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550584    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 15:00:32.553968 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550587    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 15:00:32.553968 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550590    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 15:00:32.553968 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550592    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 15:00:32.553968 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550595    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 15:00:32.554450 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550597    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 15:00:32.554450 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550600    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 15:00:32.554450 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550602    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 15:00:32.554450 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550604    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 15:00:32.554450 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550607    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 15:00:32.554450 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550609    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 15:00:32.554450 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550611    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 15:00:32.554450 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550615    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 15:00:32.554450 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550617    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 15:00:32.554450 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550620    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 15:00:32.554450 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550622    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 15:00:32.554450 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550625    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 15:00:32.554450 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550627    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 15:00:32.554450 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550630    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 15:00:32.554450 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550632    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 15:00:32.554450 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550634    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 15:00:32.554450 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550636    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 15:00:32.554450 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550639    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 15:00:32.554450 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550641    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 15:00:32.554450 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550644    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 15:00:32.555047 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550646    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 15:00:32.555047 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550648    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 15:00:32.555047 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550651    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 15:00:32.555047 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550653    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 15:00:32.555047 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550657    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 15:00:32.555047 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550660    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 15:00:32.555047 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550663    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 15:00:32.555047 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:32.550665    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 15:00:32.555047 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.550670    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 15:00:32.555047 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.550784    2567 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 07 15:00:32.555047 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.554903    2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 07 15:00:32.556611 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.556598    2567 server.go:1019] "Starting client certificate rotation"
Jul 07 15:00:32.556718 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.556701    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 15:00:32.556748 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.556734    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 15:00:32.559500 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.559487    2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 15:00:32.560716 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.560695    2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 15:00:32.576879 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.576858    2567 log.go:25] "Validated CRI v1 runtime API"
Jul 07 15:00:32.579306 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.579290    2567 log.go:25] "Validated CRI v1 image API"
Jul 07 15:00:32.580869 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.580854    2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 07 15:00:32.583470 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.583440    2567 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 80617145-112d-4333-98b3-730feb472e8d:/dev/nvme0n1p4 87407782-da4a-4687-b5bd-8e93fee51d98:/dev/nvme0n1p3]
Jul 07 15:00:32.583470 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.583459    2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 07 15:00:32.586083 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.586067    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 15:00:32.591061 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.590950    2567 manager.go:216] Machine: {Timestamp:2026-07-07 15:00:32.588948248 +0000 UTC m=+0.385484991 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099206 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec295ec06d4c6c898c537b48933392bd SystemUUID:ec295ec0-6d4c-6c89-8c53-7b48933392bd BootID:56d66c0e-d4a8-4cc1-92d9-dee107a58f51 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:de:fa:a9:57:37 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:de:fa:a9:57:37 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b2:09:f6:35:74:e0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 07 15:00:32.591061 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.591050    2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 07 15:00:32.591196 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.591131    2567 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 07 15:00:32.594618 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.594591    2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 07 15:00:32.594750 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.594620    2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-233.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 07 15:00:32.595544 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.595534    2567 topology_manager.go:138] "Creating topology manager with none policy"
Jul 07 15:00:32.595585 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.595547    2567 container_manager_linux.go:309] "Creating device plugin manager"
Jul 07 15:00:32.595585 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.595556    2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 15:00:32.595585 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.595569    2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 15:00:32.597530 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.597520    2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 07 15:00:32.601363 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.601351    2567 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 15:00:32.601481 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.601471    2567 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 07 15:00:32.601535 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.601502    2567 kubelet.go:486] "Attempting to sync node with API server"
Jul 07 15:00:32.601585 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.601539    2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 07 15:00:32.601585 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.601567    2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 07 15:00:32.601585 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.601582    2567 kubelet.go:398] "Adding apiserver pod source"
Jul 07 15:00:32.601695 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.601594    2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 07 15:00:32.602018 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.602003    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 15:00:32.602081 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.602026    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 15:00:32.604385 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.604367    2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 07 15:00:32.606941 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.606928    2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 07 15:00:32.606996 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.606958    2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 07 15:00:32.608272 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.608261    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 07 15:00:32.608272 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.608277    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 07 15:00:32.608272 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.608284    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 07 15:00:32.608272 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.608289    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 07 15:00:32.608412 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.608298    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 07 15:00:32.608412 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.608304    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 07 15:00:32.608412 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.608312    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 07 15:00:32.608412 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.608317    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 07 15:00:32.608412 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.608323    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 07 15:00:32.608412 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.608328    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 07 15:00:32.608412 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.608336    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 07 15:00:32.608412 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.608341    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 07 15:00:32.609263 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.609252    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 07 15:00:32.609263 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.609261    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 07 15:00:32.612355 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.612338    2567 server.go:1268] "Started kubelet"
Jul 07 15:00:32.612481 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.612429    2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 07 15:00:32.612549 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.612505    2567 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 07 15:00:32.612624 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.612597    2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 07 15:00:32.613159 ip-10-0-138-233 systemd[1]: Started Kubernetes Kubelet.
Jul 07 15:00:32.613288 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.613174    2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-233.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 07 15:00:32.613288 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:32.613179    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 07 15:00:32.613288 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:32.613271    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-233.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 07 15:00:32.614404 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.614389    2567 server.go:310] "Adding debug handlers to kubelet server"
Jul 07 15:00:32.614601 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.614566    2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 07 15:00:32.619123 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:32.619103    2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 07 15:00:32.621381 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:32.620314    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-233.ec2.internal.18c009fc8558c41f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-233.ec2.internal,UID:ip-10-0-138-233.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-233.ec2.internal,},FirstTimestamp:2026-07-07 15:00:32.612312095 +0000 UTC m=+0.408848814,LastTimestamp:2026-07-07 15:00:32.612312095 +0000 UTC m=+0.408848814,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-233.ec2.internal,}"
Jul 07 15:00:32.623630 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.623615    2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 07 15:00:32.623689 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.623633    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 07 15:00:32.623734 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:32.623709    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-233.ec2.internal\" not found"
Jul 07 15:00:32.624594 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.624572    2567 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 07 15:00:32.624736 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.624722    2567 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 07 15:00:32.624869 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.624560    2567 factory.go:55] Registering systemd factory
Jul 07 15:00:32.624934 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.624878    2567 factory.go:223] Registration of the systemd container factory successfully
Jul 07 15:00:32.624934 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.624570    2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 07 15:00:32.625034 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.624960    2567 reconstruct.go:97] "Volume reconstruction finished"
Jul 07 15:00:32.625034 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.624967    2567 reconciler.go:29] "Reconciler: start to sync state"
Jul 07 15:00:32.625511 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.625492    2567 factory.go:153] Registering CRI-O factory
Jul 07 15:00:32.625511 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.625512    2567 factory.go:223] Registration of the crio container factory successfully
Jul 07 15:00:32.625653 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.625560    2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 07 15:00:32.625653 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.625582    2567 factory.go:103] Registering Raw factory
Jul 07 15:00:32.625653 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.625594    2567 manager.go:1196] Started watching for new ooms in manager
Jul 07 15:00:32.632248 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.626228    2567 manager.go:349] Starting recovery of all containers
Jul 07 15:00:32.632248 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:32.627403    2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-233.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 07 15:00:32.632248 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:32.627517    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 07 15:00:32.640223 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.640090    2567 manager.go:354] Recovery completed
Jul 07 15:00:32.644314 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.644299    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 15:00:32.646316 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.646298    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 15:00:32.646391 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.646328    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 15:00:32.646391 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.646340    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientPID"
Jul 07 15:00:32.646904 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.646890    2567 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 07 15:00:32.646904 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.646902    2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 07 15:00:32.646904 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.646903    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 15:00:32.647093 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.646917    2567 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 15:00:32.647555 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.647540    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 15:00:32.647611 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.647564    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 15:00:32.647611 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.647574    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientPID"
Jul 07 15:00:32.648053 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.648032    2567 policy_none.go:49] "None policy: Start"
Jul 07 15:00:32.648053 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.648053    2567 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 07 15:00:32.648148 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.648063    2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 07 15:00:32.648675 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:32.648496    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-233.ec2.internal.18c009fc875f9d77  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-233.ec2.internal,UID:ip-10-0-138-233.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-233.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-233.ec2.internal,},FirstTimestamp:2026-07-07 15:00:32.646315383 +0000 UTC m=+0.442852102,LastTimestamp:2026-07-07 15:00:32.646315383 +0000 UTC m=+0.442852102,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-233.ec2.internal,}"
Jul 07 15:00:32.649102 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.649088    2567 policy_none.go:47] "Start"
Jul 07 15:00:32.655992 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.655974    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gczf7"
Jul 07 15:00:32.658282 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:32.658211    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-233.ec2.internal.18c009fc875fe38c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-233.ec2.internal,UID:ip-10-0-138-233.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-138-233.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-138-233.ec2.internal,},FirstTimestamp:2026-07-07 15:00:32.646333324 +0000 UTC m=+0.442870043,LastTimestamp:2026-07-07 15:00:32.646333324 +0000 UTC m=+0.442870043,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-233.ec2.internal,}"
Jul 07 15:00:32.661083 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:32.661016    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-233.ec2.internal.18c009fc87600b1e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-233.ec2.internal,UID:ip-10-0-138-233.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-138-233.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-138-233.ec2.internal,},FirstTimestamp:2026-07-07 15:00:32.646343454 +0000 UTC m=+0.442880173,LastTimestamp:2026-07-07 15:00:32.646343454 +0000 UTC m=+0.442880173,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-233.ec2.internal,}"
Jul 07 15:00:32.662223 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.662207    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gczf7"
Jul 07 15:00:32.690461 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.690445    2567 manager.go:337] "Starting Device Plugin manager"
Jul 07 15:00:32.703537 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:32.690500    2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 07 15:00:32.703537 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.690520    2567 server.go:87] "Starting device plugin registration server"
Jul 07 15:00:32.703537 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.690853    2567 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 07 15:00:32.703537 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.690866    2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 07 15:00:32.703537 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.690946    2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 07 15:00:32.703537 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.691022    2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 07 15:00:32.703537 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.691032    2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 07 15:00:32.703537 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:32.691560    2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 07 15:00:32.703537 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:32.691619    2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-233.ec2.internal\" not found"
Jul 07 15:00:32.769736 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.769696    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 07 15:00:32.771035 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.771022    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 07 15:00:32.771115 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.771061    2567 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 07 15:00:32.771115 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.771086    2567 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 07 15:00:32.771211 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:32.771117    2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 07 15:00:32.774058 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.774039    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 15:00:32.791163 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.791105    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 15:00:32.792180 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.792164    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 15:00:32.792258 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.792200    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 15:00:32.792258 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.792213    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientPID"
Jul 07 15:00:32.792258 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.792237    2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-233.ec2.internal"
Jul 07 15:00:32.805241 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.805218    2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-233.ec2.internal"
Jul 07 15:00:32.805241 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:32.805242    2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-233.ec2.internal\": node \"ip-10-0-138-233.ec2.internal\" not found"
Jul 07 15:00:32.843816 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:32.843793    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-233.ec2.internal\" not found"
Jul 07 15:00:32.871240 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.871202    2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-233.ec2.internal"]
Jul 07 15:00:32.871315 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.871288    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 15:00:32.872793 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.872778    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 15:00:32.872876 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.872809    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 15:00:32.872876 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.872825    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientPID"
Jul 07 15:00:32.875213 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.875200    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 15:00:32.875354 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.875342    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal"
Jul 07 15:00:32.875387 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.875370    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 15:00:32.875981 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.875953    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 15:00:32.875981 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.875963    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 15:00:32.876159 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.876009    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 15:00:32.876159 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.876026    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientPID"
Jul 07 15:00:32.876390 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.876376    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 15:00:32.876454 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.876396    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientPID"
Jul 07 15:00:32.878774 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.878759    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-233.ec2.internal"
Jul 07 15:00:32.878871 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.878783    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 15:00:32.879451 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.879437    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 15:00:32.879520 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.879461    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 15:00:32.879520 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.879474    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientPID"
Jul 07 15:00:32.905555 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:32.905530    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-233.ec2.internal\" not found" node="ip-10-0-138-233.ec2.internal"
Jul 07 15:00:32.909905 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:32.909888    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-233.ec2.internal\" not found" node="ip-10-0-138-233.ec2.internal"
Jul 07 15:00:32.927288 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.927266    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9a9307618ad481c92addb0219af76bb7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal\" (UID: \"9a9307618ad481c92addb0219af76bb7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal"
Jul 07 15:00:32.927356 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.927295    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a9307618ad481c92addb0219af76bb7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal\" (UID: \"9a9307618ad481c92addb0219af76bb7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal"
Jul 07 15:00:32.927356 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:32.927315    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/938243ce709c303fec6b51c38490eae8-config\") pod \"kube-apiserver-proxy-ip-10-0-138-233.ec2.internal\" (UID: \"938243ce709c303fec6b51c38490eae8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-233.ec2.internal"
Jul 07 15:00:32.944281 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:32.944253    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-233.ec2.internal\" not found"
Jul 07 15:00:33.028419 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:33.028384    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9a9307618ad481c92addb0219af76bb7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal\" (UID: \"9a9307618ad481c92addb0219af76bb7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal"
Jul 07 15:00:33.028419 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:33.028417    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a9307618ad481c92addb0219af76bb7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal\" (UID: \"9a9307618ad481c92addb0219af76bb7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal"
Jul 07 15:00:33.028612 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:33.028435    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/938243ce709c303fec6b51c38490eae8-config\") pod \"kube-apiserver-proxy-ip-10-0-138-233.ec2.internal\" (UID: \"938243ce709c303fec6b51c38490eae8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-233.ec2.internal"
Jul 07 15:00:33.028612 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:33.028463    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/938243ce709c303fec6b51c38490eae8-config\") pod \"kube-apiserver-proxy-ip-10-0-138-233.ec2.internal\" (UID: \"938243ce709c303fec6b51c38490eae8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-233.ec2.internal"
Jul 07 15:00:33.028612 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:33.028484    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a9307618ad481c92addb0219af76bb7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal\" (UID: \"9a9307618ad481c92addb0219af76bb7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal"
Jul 07 15:00:33.028612 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:33.028485    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9a9307618ad481c92addb0219af76bb7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal\" (UID: \"9a9307618ad481c92addb0219af76bb7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal"
Jul 07 15:00:33.045286 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:33.045214    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-233.ec2.internal\" not found"
Jul 07 15:00:33.146108 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:33.146073    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-233.ec2.internal\" not found"
Jul 07 15:00:33.209278 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:33.209251    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal"
Jul 07 15:00:33.212955 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:33.212932    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-233.ec2.internal"
Jul 07 15:00:33.247170 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:33.247136    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-233.ec2.internal\" not found"
Jul 07 15:00:33.347817 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:33.347726    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-233.ec2.internal\" not found"
Jul 07 15:00:33.430887 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:33.430856    2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 15:00:33.448232 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:33.448209    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-233.ec2.internal\" not found"
Jul 07 15:00:33.548799 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:33.548759    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-233.ec2.internal\" not found"
Jul 07 15:00:33.555950 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:33.555918    2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 07 15:00:33.556109 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:33.556089    2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 15:00:33.556168 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:33.556122    2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 15:00:33.566555 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:33.566531    2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 15:00:33.649528 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:33.649439    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-233.ec2.internal\" not found"
Jul 07 15:00:33.664630 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:33.664598    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 14:55:33 +0000 UTC" deadline="2028-04-02 04:03:57.273262858 +0000 UTC"
Jul 07 15:00:33.664630 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:33.664627    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15229h3m23.608639373s"
Jul 07 15:00:33.750296 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:33.750270    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-233.ec2.internal\" not found"
Jul 07 15:00:33.812740 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:33.812714    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 15:00:33.814138 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:33.814117    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 15:00:33.814259 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:33.814160    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 15:00:33.814259 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:33.814176    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeHasSufficientPID"
Jul 07 15:00:33.850412 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:33.850385    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-233.ec2.internal\" not found"
Jul 07 15:00:33.863487 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:33.863454    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod938243ce709c303fec6b51c38490eae8.slice/crio-fe41f197106b84e282602198dbae092063a016be5b62f3f6bd62f7c9607ccb82 WatchSource:0}: Error finding container fe41f197106b84e282602198dbae092063a016be5b62f3f6bd62f7c9607ccb82: Status 404 returned error can't find the container with id fe41f197106b84e282602198dbae092063a016be5b62f3f6bd62f7c9607ccb82
Jul 07 15:00:33.863883 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:33.863857    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a9307618ad481c92addb0219af76bb7.slice/crio-1f78d828ea3aab78777ac634822585ce90f43b90007916c30f46703a117add58 WatchSource:0}: Error finding container 1f78d828ea3aab78777ac634822585ce90f43b90007916c30f46703a117add58: Status 404 returned error can't find the container with id 1f78d828ea3aab78777ac634822585ce90f43b90007916c30f46703a117add58
Jul 07 15:00:33.869592 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:33.869577    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 15:00:33.951144 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:33.951063    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-233.ec2.internal\" not found"
Jul 07 15:00:34.051682 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:34.051647    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-233.ec2.internal\" not found"
Jul 07 15:00:34.112357 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.112328    2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 15:00:34.124355 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.124332    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal"
Jul 07 15:00:34.143865 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.143845    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 15:00:34.144610 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.144596    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-233.ec2.internal"
Jul 07 15:00:34.157342 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.157325    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 15:00:34.562526 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.562435    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 15:00:34.602570 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.602539    2567 apiserver.go:52] "Watching apiserver"
Jul 07 15:00:34.613322 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.613290    2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 07 15:00:34.614529 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.614501    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-9gmwd","openshift-network-operator/iptables-alerter-8n5q4","openshift-ovn-kubernetes/ovnkube-node-xj52l","kube-system/global-pull-secret-syncer-zz5ql","kube-system/kube-apiserver-proxy-ip-10-0-138-233.ec2.internal","openshift-cluster-node-tuning-operator/tuned-vgrxt","openshift-dns/node-resolver-d2br4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal","openshift-multus/network-metrics-daemon-zs477","kube-system/konnectivity-agent-7tg7l","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb","openshift-image-registry/node-ca-zk8hk","openshift-multus/multus-additional-cni-plugins-5r27h","openshift-multus/multus-z8276"]
Jul 07 15:00:34.617477 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.617451    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-d2br4"
Jul 07 15:00:34.620949 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.620450    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb"
Jul 07 15:00:34.621243 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.621072    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 07 15:00:34.621243 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.621069    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4ptwx\""
Jul 07 15:00:34.621243 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.621097    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 07 15:00:34.622647 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.622625    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.623549 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.623526    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 07 15:00:34.623664 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.623579    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 07 15:00:34.623664 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.623601    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 07 15:00:34.623839 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.623798    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hzdc8\""
Jul 07 15:00:34.624461 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.624444    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 07 15:00:34.628198 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.628176    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 07 15:00:34.628300 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.628234    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 07 15:00:34.628442 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.628425    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-x8xkk\""
Jul 07 15:00:34.628579 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.628565    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 07 15:00:34.628679 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.628640    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 07 15:00:34.628752 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.628696    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 07 15:00:34.628819 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.628774    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:00:34.628883 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:34.628853    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zz5ql" podUID="31b4f468-dc43-4504-b8c1-3f6b91c08bc5"
Jul 07 15:00:34.629350 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.629004    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.630342 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.630327    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 07 15:00:34.631239 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.631219    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.633945 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.633807    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:00:34.633945 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.633883    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 07 15:00:34.633945 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:34.633901    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zs477" podUID="0ebbf4a6-429c-465d-916d-84567a210db5"
Jul 07 15:00:34.638846 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.638805    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xcr2x\""
Jul 07 15:00:34.639011 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.638992    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 07 15:00:34.639078 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.639008    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 07 15:00:34.639270 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.639248    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6hj2w\""
Jul 07 15:00:34.639474 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.638822    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 07 15:00:34.639540 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.639482    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 07 15:00:34.639981 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.639959    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-original-pull-secret\") pod \"global-pull-secret-syncer-zz5ql\" (UID: \"31b4f468-dc43-4504-b8c1-3f6b91c08bc5\") " pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:00:34.639981 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.639977    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5r27h"
Jul 07 15:00:34.640129 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.640006    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-cnibin\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.640129 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.640043    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-host-run-k8s-cni-cncf-io\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.640129 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.640076    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4017287a-0129-4a52-8875-045408b9a82d-hosts-file\") pod \"node-resolver-d2br4\" (UID: \"4017287a-0129-4a52-8875-045408b9a82d\") " pod="openshift-dns/node-resolver-d2br4"
Jul 07 15:00:34.640129 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.640105    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-host-run-netns\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.640129 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.640126    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 07 15:00:34.640382 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.640161    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-etc-sysconfig\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.640382 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.640244    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-etc-sysctl-d\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.640382 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.640275    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b18f878a-7e28-4cb3-92b9-521847f10288-etc-tuned\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.640382 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.640309    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-host-run-netns\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.640382 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.640336    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ca0a1db1-0575-4ea6-a93a-6445a39114b2-ovnkube-config\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.640382 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.640367    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b0751af-98d6-4f05-ae7f-d1cff06cf817-cni-binary-copy\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.640676 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.640400    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8n5q4"
Jul 07 15:00:34.640676 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.640401    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-multus-conf-dir\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.640676 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.640471    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-host-run-ovn-kubernetes\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.640676 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.640504    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-os-release\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.640676 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.640545    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-etc-modprobe-d\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.640676 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.640580    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-etc-sysctl-conf\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.640864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.640611    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-lib-modules\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.640864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.640740    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-var-lib-openvswitch\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.640864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.640774    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-run-ovn\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.640864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.640801    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-system-cni-dir\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.640864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.640833    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-etc-kubernetes\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.641121 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.640864    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-run\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.641121 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.640897    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wq95\" (UniqueName: \"kubernetes.io/projected/8b0751af-98d6-4f05-ae7f-d1cff06cf817-kube-api-access-2wq95\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.641121 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.640948    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-node-log\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.641438 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.641389    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-host-cni-bin\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.641511 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.641457    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c7006a3c-1f66-4a34-8ae8-915722738e79-sys-fs\") pod \"aws-ebs-csi-driver-node-nbwtb\" (UID: \"c7006a3c-1f66-4a34-8ae8-915722738e79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb"
Jul 07 15:00:34.641625 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.641605    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-host-kubelet\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.641704 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.641643    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c7006a3c-1f66-4a34-8ae8-915722738e79-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nbwtb\" (UID: \"c7006a3c-1f66-4a34-8ae8-915722738e79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb"
Jul 07 15:00:34.641704 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.641664    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-var-lib-kubelet\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.641704 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.641693    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c7006a3c-1f66-4a34-8ae8-915722738e79-socket-dir\") pod \"aws-ebs-csi-driver-node-nbwtb\" (UID: \"c7006a3c-1f66-4a34-8ae8-915722738e79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb"
Jul 07 15:00:34.641835 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.641719    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdjvc\" (UniqueName: \"kubernetes.io/projected/c7006a3c-1f66-4a34-8ae8-915722738e79-kube-api-access-mdjvc\") pod \"aws-ebs-csi-driver-node-nbwtb\" (UID: \"c7006a3c-1f66-4a34-8ae8-915722738e79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb"
Jul 07 15:00:34.641835 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.641750    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-run-systemd\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.641835 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.641780    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ca0a1db1-0575-4ea6-a93a-6445a39114b2-ovnkube-script-lib\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.641835 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.641809    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-host-var-lib-kubelet\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.642043 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.641836    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-etc-kubernetes\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.642043 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.641918    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c7006a3c-1f66-4a34-8ae8-915722738e79-etc-selinux\") pod \"aws-ebs-csi-driver-node-nbwtb\" (UID: \"c7006a3c-1f66-4a34-8ae8-915722738e79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb"
Jul 07 15:00:34.642043 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.641950    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-host-cni-netd\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.642043 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.641974    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-multus-cni-dir\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.642043 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.642006    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-multus-socket-dir-parent\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.642884 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.642036    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b18f878a-7e28-4cb3-92b9-521847f10288-tmp\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.642996 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.642932    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svk4q\" (UniqueName: \"kubernetes.io/projected/b18f878a-7e28-4cb3-92b9-521847f10288-kube-api-access-svk4q\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.642996 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.642961    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c7006a3c-1f66-4a34-8ae8-915722738e79-device-dir\") pod \"aws-ebs-csi-driver-node-nbwtb\" (UID: \"c7006a3c-1f66-4a34-8ae8-915722738e79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb"
Jul 07 15:00:34.642996 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.642987    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ca0a1db1-0575-4ea6-a93a-6445a39114b2-env-overrides\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.643124 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.643077    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-etc-openvswitch\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.643124 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.643116    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv77h\" (UniqueName: \"kubernetes.io/projected/4017287a-0129-4a52-8875-045408b9a82d-kube-api-access-jv77h\") pod \"node-resolver-d2br4\" (UID: \"4017287a-0129-4a52-8875-045408b9a82d\") " pod="openshift-dns/node-resolver-d2br4"
Jul 07 15:00:34.643230 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.643160    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-hostroot\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.643230 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.643198    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-host\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.643230 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.643225    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ca0a1db1-0575-4ea6-a93a-6445a39114b2-ovn-node-metrics-cert\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.643380 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.643245    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 07 15:00:34.643380 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.643251    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-dbus\") pod \"global-pull-secret-syncer-zz5ql\" (UID: \"31b4f468-dc43-4504-b8c1-3f6b91c08bc5\") " pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:00:34.643380 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.643277    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-etc-systemd\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.643380 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.643301    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-sys\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.643380 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.643325    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-host-slash\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.643380 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.643350    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-kubelet-config\") pod \"global-pull-secret-syncer-zz5ql\" (UID: \"31b4f468-dc43-4504-b8c1-3f6b91c08bc5\") " pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:00:34.643380 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.643373    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-host-run-multus-certs\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.643717 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.643397    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-log-socket\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.643717 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.643421    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4h56\" (UniqueName: \"kubernetes.io/projected/ca0a1db1-0575-4ea6-a93a-6445a39114b2-kube-api-access-k4h56\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.643717 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.643442    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-host-var-lib-cni-bin\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.643717 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.643496    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.643717 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.643537    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c7006a3c-1f66-4a34-8ae8-915722738e79-registration-dir\") pod \"aws-ebs-csi-driver-node-nbwtb\" (UID: \"c7006a3c-1f66-4a34-8ae8-915722738e79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb"
Jul 07 15:00:34.643717 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.643568    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-systemd-units\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.643717 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.643595    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-host-var-lib-cni-multus\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.643717 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.643620    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8b0751af-98d6-4f05-ae7f-d1cff06cf817-multus-daemon-config\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.643717 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.643650    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4017287a-0129-4a52-8875-045408b9a82d-tmp-dir\") pod \"node-resolver-d2br4\" (UID: \"4017287a-0129-4a52-8875-045408b9a82d\") " pod="openshift-dns/node-resolver-d2br4"
Jul 07 15:00:34.643717 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.643674    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-run-openvswitch\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.644184 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.643822    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-tfmx7\""
Jul 07 15:00:34.644184 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.643979    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 07 15:00:34.644184 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.644082    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zdfqh\""
Jul 07 15:00:34.644184 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.644156    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 07 15:00:34.645035 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.645017    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7tg7l"
Jul 07 15:00:34.645124 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.645083    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 07 15:00:34.645181 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.645101    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 15:00:34.645181 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.645173    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 07 15:00:34.648287 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.648265    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 07 15:00:34.648377 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.648293    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 07 15:00:34.648377 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.648355    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:00:34.648461 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:34.648423    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gmwd" podUID="a7a85a73-907b-4a3b-bee4-b03ce0da2b6c"
Jul 07 15:00:34.648537 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.648519    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zqgpb\""
Jul 07 15:00:34.650539 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.650520    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zk8hk"
Jul 07 15:00:34.653538 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.653512    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vg527\""
Jul 07 15:00:34.653538 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.653534    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 07 15:00:34.653702 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.653592    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 07 15:00:34.653702 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.653539    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 07 15:00:34.726256 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.726225    2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 07 15:00:34.744850 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.744801    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-host-var-lib-cni-multus\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.744850 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.744836    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8b0751af-98d6-4f05-ae7f-d1cff06cf817-multus-daemon-config\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.744850 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.744880    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/671cc2c7-4b8d-4183-8a74-16d971f3431e-agent-certs\") pod \"konnectivity-agent-7tg7l\" (UID: \"671cc2c7-4b8d-4183-8a74-16d971f3431e\") " pod="kube-system/konnectivity-agent-7tg7l"
Jul 07 15:00:34.745173 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.744906    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4017287a-0129-4a52-8875-045408b9a82d-tmp-dir\") pod \"node-resolver-d2br4\" (UID: \"4017287a-0129-4a52-8875-045408b9a82d\") " pod="openshift-dns/node-resolver-d2br4"
Jul 07 15:00:34.745173 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.744925    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-host-var-lib-cni-multus\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.745173 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.744926    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-run-openvswitch\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.745173 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.744980    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-run-openvswitch\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.745173 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745005    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-original-pull-secret\") pod \"global-pull-secret-syncer-zz5ql\" (UID: \"31b4f468-dc43-4504-b8c1-3f6b91c08bc5\") " pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:00:34.745173 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745035    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-cnibin\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.745173 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745062    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-host-run-k8s-cni-cncf-io\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.745173 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745086    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4017287a-0129-4a52-8875-045408b9a82d-hosts-file\") pod \"node-resolver-d2br4\" (UID: \"4017287a-0129-4a52-8875-045408b9a82d\") " pod="openshift-dns/node-resolver-d2br4"
Jul 07 15:00:34.745173 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745114    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-host-run-netns\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.745173 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745139    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-etc-sysconfig\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.745173 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745166    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-etc-sysctl-d\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.745637 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:34.745178    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 15:00:34.745637 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745261    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4017287a-0129-4a52-8875-045408b9a82d-tmp-dir\") pod \"node-resolver-d2br4\" (UID: \"4017287a-0129-4a52-8875-045408b9a82d\") " pod="openshift-dns/node-resolver-d2br4"
Jul 07 15:00:34.745637 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:34.745265    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-original-pull-secret podName:31b4f468-dc43-4504-b8c1-3f6b91c08bc5 nodeName:}" failed. No retries permitted until 2026-07-07 15:00:35.245240812 +0000 UTC m=+3.041777538 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-original-pull-secret") pod "global-pull-secret-syncer-zz5ql" (UID: "31b4f468-dc43-4504-b8c1-3f6b91c08bc5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 15:00:34.745637 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745330    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4017287a-0129-4a52-8875-045408b9a82d-hosts-file\") pod \"node-resolver-d2br4\" (UID: \"4017287a-0129-4a52-8875-045408b9a82d\") " pod="openshift-dns/node-resolver-d2br4"
Jul 07 15:00:34.745637 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745366    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-cnibin\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.745637 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745393    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-host-run-k8s-cni-cncf-io\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.745637 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745185    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b18f878a-7e28-4cb3-92b9-521847f10288-etc-tuned\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.745637 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745413    2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 07 15:00:34.745637 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745473    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-etc-sysconfig\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.745637 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745494    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-host-run-netns\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.745637 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745428    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c6eef884-173e-4c5b-890d-3409f42fb0a3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5r27h\" (UID: \"c6eef884-173e-4c5b-890d-3409f42fb0a3\") " pod="openshift-multus/multus-additional-cni-plugins-5r27h"
Jul 07 15:00:34.745637 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745537    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/88f0b024-94cb-44b4-a5f4-9497ce96c3cb-host\") pod \"node-ca-zk8hk\" (UID: \"88f0b024-94cb-44b4-a5f4-9497ce96c3cb\") " pod="openshift-image-registry/node-ca-zk8hk"
Jul 07 15:00:34.745637 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745557    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-etc-sysctl-d\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.745637 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745569    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8b0751af-98d6-4f05-ae7f-d1cff06cf817-multus-daemon-config\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.745637 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745574    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-host-run-netns\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.745637 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745617    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-host-run-netns\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.745637 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745633    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ca0a1db1-0575-4ea6-a93a-6445a39114b2-ovnkube-config\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.746692 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745662    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b0751af-98d6-4f05-ae7f-d1cff06cf817-cni-binary-copy\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.746692 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745687    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-multus-conf-dir\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.746692 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745719    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnch2\" (UniqueName: \"kubernetes.io/projected/88f0b024-94cb-44b4-a5f4-9497ce96c3cb-kube-api-access-nnch2\") pod \"node-ca-zk8hk\" (UID: \"88f0b024-94cb-44b4-a5f4-9497ce96c3cb\") " pod="openshift-image-registry/node-ca-zk8hk"
Jul 07 15:00:34.746692 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745748    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-host-run-ovn-kubernetes\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.746692 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745774    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-os-release\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.746692 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745801    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-etc-modprobe-d\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.746692 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745827    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-etc-sysctl-conf\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.746692 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745854    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-lib-modules\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.746692 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745879    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c6eef884-173e-4c5b-890d-3409f42fb0a3-os-release\") pod \"multus-additional-cni-plugins-5r27h\" (UID: \"c6eef884-173e-4c5b-890d-3409f42fb0a3\") " pod="openshift-multus/multus-additional-cni-plugins-5r27h"
Jul 07 15:00:34.746692 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745932    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c6eef884-173e-4c5b-890d-3409f42fb0a3-cni-binary-copy\") pod \"multus-additional-cni-plugins-5r27h\" (UID: \"c6eef884-173e-4c5b-890d-3409f42fb0a3\") " pod="openshift-multus/multus-additional-cni-plugins-5r27h"
Jul 07 15:00:34.746692 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745961    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvn2q\" (UniqueName: \"kubernetes.io/projected/a7a85a73-907b-4a3b-bee4-b03ce0da2b6c-kube-api-access-jvn2q\") pod \"network-check-target-9gmwd\" (UID: \"a7a85a73-907b-4a3b-bee4-b03ce0da2b6c\") " pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:00:34.746692 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.745989    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-var-lib-openvswitch\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.746692 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746014    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-run-ovn\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.746692 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746039    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-system-cni-dir\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.746692 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746068    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-etc-kubernetes\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.746692 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746093    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-run\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.746692 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746120    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c6eef884-173e-4c5b-890d-3409f42fb0a3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5r27h\" (UID: \"c6eef884-173e-4c5b-890d-3409f42fb0a3\") " pod="openshift-multus/multus-additional-cni-plugins-5r27h"
Jul 07 15:00:34.747729 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746149    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/984f3e1f-0860-444a-bb2e-454b5d450217-iptables-alerter-script\") pod \"iptables-alerter-8n5q4\" (UID: \"984f3e1f-0860-444a-bb2e-454b5d450217\") " pod="openshift-network-operator/iptables-alerter-8n5q4"
Jul 07 15:00:34.747729 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746173    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/984f3e1f-0860-444a-bb2e-454b5d450217-host-slash\") pod \"iptables-alerter-8n5q4\" (UID: \"984f3e1f-0860-444a-bb2e-454b5d450217\") " pod="openshift-network-operator/iptables-alerter-8n5q4"
Jul 07 15:00:34.747729 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746201    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b0751af-98d6-4f05-ae7f-d1cff06cf817-cni-binary-copy\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.747729 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746200    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/671cc2c7-4b8d-4183-8a74-16d971f3431e-konnectivity-ca\") pod \"konnectivity-agent-7tg7l\" (UID: \"671cc2c7-4b8d-4183-8a74-16d971f3431e\") " pod="kube-system/konnectivity-agent-7tg7l"
Jul 07 15:00:34.747729 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746262    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wq95\" (UniqueName: \"kubernetes.io/projected/8b0751af-98d6-4f05-ae7f-d1cff06cf817-kube-api-access-2wq95\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.747729 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746283    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ca0a1db1-0575-4ea6-a93a-6445a39114b2-ovnkube-config\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.747729 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746287    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-multus-conf-dir\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.747729 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746290    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-node-log\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.747729 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746332    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-node-log\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.747729 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746383    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-host-run-ovn-kubernetes\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.747729 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746396    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-var-lib-openvswitch\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.747729 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746417    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-etc-modprobe-d\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.747729 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746440    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-run-ovn\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.747729 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746442    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-os-release\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.747729 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746445    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-host-cni-bin\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.747729 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746476    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-host-cni-bin\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.747729 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746492    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c7006a3c-1f66-4a34-8ae8-915722738e79-sys-fs\") pod \"aws-ebs-csi-driver-node-nbwtb\" (UID: \"c7006a3c-1f66-4a34-8ae8-915722738e79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb"
Jul 07 15:00:34.747729 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746530    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-host-kubelet\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.748974 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746541    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-lib-modules\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.748974 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746560    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c7006a3c-1f66-4a34-8ae8-915722738e79-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nbwtb\" (UID: \"c7006a3c-1f66-4a34-8ae8-915722738e79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb"
Jul 07 15:00:34.748974 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746588    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-var-lib-kubelet\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.748974 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746639    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c6eef884-173e-4c5b-890d-3409f42fb0a3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5r27h\" (UID: \"c6eef884-173e-4c5b-890d-3409f42fb0a3\") " pod="openshift-multus/multus-additional-cni-plugins-5r27h"
Jul 07 15:00:34.748974 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746648    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-etc-sysctl-conf\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.748974 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746675    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c7006a3c-1f66-4a34-8ae8-915722738e79-socket-dir\") pod \"aws-ebs-csi-driver-node-nbwtb\" (UID: \"c7006a3c-1f66-4a34-8ae8-915722738e79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb"
Jul 07 15:00:34.748974 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746693    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-system-cni-dir\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.748974 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746713    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdjvc\" (UniqueName: \"kubernetes.io/projected/c7006a3c-1f66-4a34-8ae8-915722738e79-kube-api-access-mdjvc\") pod \"aws-ebs-csi-driver-node-nbwtb\" (UID: \"c7006a3c-1f66-4a34-8ae8-915722738e79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb"
Jul 07 15:00:34.748974 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746711    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-host-kubelet\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.748974 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746737    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-etc-kubernetes\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.748974 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746742    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-run-systemd\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.748974 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746777    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ca0a1db1-0575-4ea6-a93a-6445a39114b2-ovnkube-script-lib\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.748974 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746780    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-var-lib-kubelet\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.748974 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746781    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-run\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.748974 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746809    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c7006a3c-1f66-4a34-8ae8-915722738e79-sys-fs\") pod \"aws-ebs-csi-driver-node-nbwtb\" (UID: \"c7006a3c-1f66-4a34-8ae8-915722738e79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb"
Jul 07 15:00:34.748974 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746851    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c7006a3c-1f66-4a34-8ae8-915722738e79-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nbwtb\" (UID: \"c7006a3c-1f66-4a34-8ae8-915722738e79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb"
Jul 07 15:00:34.748974 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746895    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-host-var-lib-kubelet\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.750261 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746910    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c7006a3c-1f66-4a34-8ae8-915722738e79-socket-dir\") pod \"aws-ebs-csi-driver-node-nbwtb\" (UID: \"c7006a3c-1f66-4a34-8ae8-915722738e79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb"
Jul 07 15:00:34.750261 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746953    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-run-systemd\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.750261 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746960    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-etc-kubernetes\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.750261 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.746991    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ebbf4a6-429c-465d-916d-84567a210db5-metrics-certs\") pod \"network-metrics-daemon-zs477\" (UID: \"0ebbf4a6-429c-465d-916d-84567a210db5\") " pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:00:34.750261 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747044    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-host-var-lib-kubelet\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.750261 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747077    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwxrk\" (UniqueName: \"kubernetes.io/projected/0ebbf4a6-429c-465d-916d-84567a210db5-kube-api-access-cwxrk\") pod \"network-metrics-daemon-zs477\" (UID: \"0ebbf4a6-429c-465d-916d-84567a210db5\") " pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:00:34.750261 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747092    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-etc-kubernetes\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.750261 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747137    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c7006a3c-1f66-4a34-8ae8-915722738e79-etc-selinux\") pod \"aws-ebs-csi-driver-node-nbwtb\" (UID: \"c7006a3c-1f66-4a34-8ae8-915722738e79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb"
Jul 07 15:00:34.750261 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747171    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-host-cni-netd\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.750261 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747201    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-multus-cni-dir\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.750261 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747235    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-multus-socket-dir-parent\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.750261 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747267    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-host-cni-netd\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.750261 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747286    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b18f878a-7e28-4cb3-92b9-521847f10288-tmp\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.750261 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747277    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c7006a3c-1f66-4a34-8ae8-915722738e79-etc-selinux\") pod \"aws-ebs-csi-driver-node-nbwtb\" (UID: \"c7006a3c-1f66-4a34-8ae8-915722738e79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb"
Jul 07 15:00:34.750261 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747310    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svk4q\" (UniqueName: \"kubernetes.io/projected/b18f878a-7e28-4cb3-92b9-521847f10288-kube-api-access-svk4q\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.750261 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747331    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ca0a1db1-0575-4ea6-a93a-6445a39114b2-ovnkube-script-lib\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.750261 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747347    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c7006a3c-1f66-4a34-8ae8-915722738e79-device-dir\") pod \"aws-ebs-csi-driver-node-nbwtb\" (UID: \"c7006a3c-1f66-4a34-8ae8-915722738e79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb"
Jul 07 15:00:34.751190 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747383    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c7006a3c-1f66-4a34-8ae8-915722738e79-device-dir\") pod \"aws-ebs-csi-driver-node-nbwtb\" (UID: \"c7006a3c-1f66-4a34-8ae8-915722738e79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb"
Jul 07 15:00:34.751190 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747439    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ca0a1db1-0575-4ea6-a93a-6445a39114b2-env-overrides\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.751190 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747464    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-multus-cni-dir\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.751190 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747473    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-etc-openvswitch\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.751190 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747508    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-multus-socket-dir-parent\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.751190 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747542    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv77h\" (UniqueName: \"kubernetes.io/projected/4017287a-0129-4a52-8875-045408b9a82d-kube-api-access-jv77h\") pod \"node-resolver-d2br4\" (UID: \"4017287a-0129-4a52-8875-045408b9a82d\") " pod="openshift-dns/node-resolver-d2br4"
Jul 07 15:00:34.751190 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747559    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-hostroot\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.751190 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747583    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-host\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.751190 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747626    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-hostroot\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.751190 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747630    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-etc-openvswitch\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.751190 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747657    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/88f0b024-94cb-44b4-a5f4-9497ce96c3cb-serviceca\") pod \"node-ca-zk8hk\" (UID: \"88f0b024-94cb-44b4-a5f4-9497ce96c3cb\") " pod="openshift-image-registry/node-ca-zk8hk"
Jul 07 15:00:34.751190 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747689    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ca0a1db1-0575-4ea6-a93a-6445a39114b2-ovn-node-metrics-cert\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.751190 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747724    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-dbus\") pod \"global-pull-secret-syncer-zz5ql\" (UID: \"31b4f468-dc43-4504-b8c1-3f6b91c08bc5\") " pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:00:34.751190 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747747    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-etc-systemd\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.751190 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747766    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-sys\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.751190 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747790    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-host-slash\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.751190 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747835    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-kubelet-config\") pod \"global-pull-secret-syncer-zz5ql\" (UID: \"31b4f468-dc43-4504-b8c1-3f6b91c08bc5\") " pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:00:34.751190 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747894    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-host-slash\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.751809 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747958    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-host-run-multus-certs\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.751809 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747985    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-etc-systemd\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.751809 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.748052    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-sys\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.751809 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.748062    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-dbus\") pod \"global-pull-secret-syncer-zz5ql\" (UID: \"31b4f468-dc43-4504-b8c1-3f6b91c08bc5\") " pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:00:34.751809 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.748105    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b18f878a-7e28-4cb3-92b9-521847f10288-host\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.751809 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.748112    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-kubelet-config\") pod \"global-pull-secret-syncer-zz5ql\" (UID: \"31b4f468-dc43-4504-b8c1-3f6b91c08bc5\") " pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:00:34.751809 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.748124    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ca0a1db1-0575-4ea6-a93a-6445a39114b2-env-overrides\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.751809 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.747886    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-host-run-multus-certs\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.751809 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.748181    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c6eef884-173e-4c5b-890d-3409f42fb0a3-system-cni-dir\") pod \"multus-additional-cni-plugins-5r27h\" (UID: \"c6eef884-173e-4c5b-890d-3409f42fb0a3\") " pod="openshift-multus/multus-additional-cni-plugins-5r27h"
Jul 07 15:00:34.751809 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.748221    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdg6q\" (UniqueName: \"kubernetes.io/projected/c6eef884-173e-4c5b-890d-3409f42fb0a3-kube-api-access-xdg6q\") pod \"multus-additional-cni-plugins-5r27h\" (UID: \"c6eef884-173e-4c5b-890d-3409f42fb0a3\") " pod="openshift-multus/multus-additional-cni-plugins-5r27h"
Jul 07 15:00:34.751809 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.748260    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcsmp\" (UniqueName: \"kubernetes.io/projected/984f3e1f-0860-444a-bb2e-454b5d450217-kube-api-access-hcsmp\") pod \"iptables-alerter-8n5q4\" (UID: \"984f3e1f-0860-444a-bb2e-454b5d450217\") " pod="openshift-network-operator/iptables-alerter-8n5q4"
Jul 07 15:00:34.751809 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.748293    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-log-socket\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.751809 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.748324    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4h56\" (UniqueName: \"kubernetes.io/projected/ca0a1db1-0575-4ea6-a93a-6445a39114b2-kube-api-access-k4h56\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.751809 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.748373    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-log-socket\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.751809 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.748402    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-host-var-lib-cni-bin\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.751809 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.748529    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c6eef884-173e-4c5b-890d-3409f42fb0a3-cnibin\") pod \"multus-additional-cni-plugins-5r27h\" (UID: \"c6eef884-173e-4c5b-890d-3409f42fb0a3\") " pod="openshift-multus/multus-additional-cni-plugins-5r27h"
Jul 07 15:00:34.751809 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.748560    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.752644 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.748595    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c7006a3c-1f66-4a34-8ae8-915722738e79-registration-dir\") pod \"aws-ebs-csi-driver-node-nbwtb\" (UID: \"c7006a3c-1f66-4a34-8ae8-915722738e79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb"
Jul 07 15:00:34.752644 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.748666    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8b0751af-98d6-4f05-ae7f-d1cff06cf817-host-var-lib-cni-bin\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.752644 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.748682    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-systemd-units\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.752644 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.748729    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.752644 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.748749    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c7006a3c-1f66-4a34-8ae8-915722738e79-registration-dir\") pod \"aws-ebs-csi-driver-node-nbwtb\" (UID: \"c7006a3c-1f66-4a34-8ae8-915722738e79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb"
Jul 07 15:00:34.752644 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.748806    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ca0a1db1-0575-4ea6-a93a-6445a39114b2-systemd-units\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.752644 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.749596    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b18f878a-7e28-4cb3-92b9-521847f10288-etc-tuned\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.752644 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.751294    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b18f878a-7e28-4cb3-92b9-521847f10288-tmp\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.752644 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.751750    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ca0a1db1-0575-4ea6-a93a-6445a39114b2-ovn-node-metrics-cert\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.760195 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.760133    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wq95\" (UniqueName: \"kubernetes.io/projected/8b0751af-98d6-4f05-ae7f-d1cff06cf817-kube-api-access-2wq95\") pod \"multus-z8276\" (UID: \"8b0751af-98d6-4f05-ae7f-d1cff06cf817\") " pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.762622 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.762553    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4h56\" (UniqueName: \"kubernetes.io/projected/ca0a1db1-0575-4ea6-a93a-6445a39114b2-kube-api-access-k4h56\") pod \"ovnkube-node-xj52l\" (UID: \"ca0a1db1-0575-4ea6-a93a-6445a39114b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.763107 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.763062    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svk4q\" (UniqueName: \"kubernetes.io/projected/b18f878a-7e28-4cb3-92b9-521847f10288-kube-api-access-svk4q\") pod \"tuned-vgrxt\" (UID: \"b18f878a-7e28-4cb3-92b9-521847f10288\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.765300 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.765253    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv77h\" (UniqueName: \"kubernetes.io/projected/4017287a-0129-4a52-8875-045408b9a82d-kube-api-access-jv77h\") pod \"node-resolver-d2br4\" (UID: \"4017287a-0129-4a52-8875-045408b9a82d\") " pod="openshift-dns/node-resolver-d2br4"
Jul 07 15:00:34.768710 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.768683    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdjvc\" (UniqueName: \"kubernetes.io/projected/c7006a3c-1f66-4a34-8ae8-915722738e79-kube-api-access-mdjvc\") pod \"aws-ebs-csi-driver-node-nbwtb\" (UID: \"c7006a3c-1f66-4a34-8ae8-915722738e79\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb"
Jul 07 15:00:34.771990 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.771967    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9282s"
Jul 07 15:00:34.777184 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.777122    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal" event={"ID":"9a9307618ad481c92addb0219af76bb7","Type":"ContainerStarted","Data":"1f78d828ea3aab78777ac634822585ce90f43b90007916c30f46703a117add58"}
Jul 07 15:00:34.778484 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.778456    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-233.ec2.internal" event={"ID":"938243ce709c303fec6b51c38490eae8","Type":"ContainerStarted","Data":"fe41f197106b84e282602198dbae092063a016be5b62f3f6bd62f7c9607ccb82"}
Jul 07 15:00:34.784709 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.784683    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9282s"
Jul 07 15:00:34.849190 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.849099    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/671cc2c7-4b8d-4183-8a74-16d971f3431e-agent-certs\") pod \"konnectivity-agent-7tg7l\" (UID: \"671cc2c7-4b8d-4183-8a74-16d971f3431e\") " pod="kube-system/konnectivity-agent-7tg7l"
Jul 07 15:00:34.849190 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.849158    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c6eef884-173e-4c5b-890d-3409f42fb0a3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5r27h\" (UID: \"c6eef884-173e-4c5b-890d-3409f42fb0a3\") " pod="openshift-multus/multus-additional-cni-plugins-5r27h"
Jul 07 15:00:34.849478 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.849198    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/88f0b024-94cb-44b4-a5f4-9497ce96c3cb-host\") pod \"node-ca-zk8hk\" (UID: \"88f0b024-94cb-44b4-a5f4-9497ce96c3cb\") " pod="openshift-image-registry/node-ca-zk8hk"
Jul 07 15:00:34.849478 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.849237    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnch2\" (UniqueName: \"kubernetes.io/projected/88f0b024-94cb-44b4-a5f4-9497ce96c3cb-kube-api-access-nnch2\") pod \"node-ca-zk8hk\" (UID: \"88f0b024-94cb-44b4-a5f4-9497ce96c3cb\") " pod="openshift-image-registry/node-ca-zk8hk"
Jul 07 15:00:34.849478 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.849291    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/88f0b024-94cb-44b4-a5f4-9497ce96c3cb-host\") pod \"node-ca-zk8hk\" (UID: \"88f0b024-94cb-44b4-a5f4-9497ce96c3cb\") " pod="openshift-image-registry/node-ca-zk8hk"
Jul 07 15:00:34.849478 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.849327    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c6eef884-173e-4c5b-890d-3409f42fb0a3-os-release\") pod \"multus-additional-cni-plugins-5r27h\" (UID: \"c6eef884-173e-4c5b-890d-3409f42fb0a3\") " pod="openshift-multus/multus-additional-cni-plugins-5r27h"
Jul 07 15:00:34.849478 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.849399    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c6eef884-173e-4c5b-890d-3409f42fb0a3-os-release\") pod \"multus-additional-cni-plugins-5r27h\" (UID: \"c6eef884-173e-4c5b-890d-3409f42fb0a3\") " pod="openshift-multus/multus-additional-cni-plugins-5r27h"
Jul 07 15:00:34.849478 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.849427    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c6eef884-173e-4c5b-890d-3409f42fb0a3-cni-binary-copy\") pod \"multus-additional-cni-plugins-5r27h\" (UID: \"c6eef884-173e-4c5b-890d-3409f42fb0a3\") " pod="openshift-multus/multus-additional-cni-plugins-5r27h"
Jul 07 15:00:34.849478 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.849454    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvn2q\" (UniqueName: \"kubernetes.io/projected/a7a85a73-907b-4a3b-bee4-b03ce0da2b6c-kube-api-access-jvn2q\") pod \"network-check-target-9gmwd\" (UID: \"a7a85a73-907b-4a3b-bee4-b03ce0da2b6c\") " pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:00:34.849837 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.849484    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c6eef884-173e-4c5b-890d-3409f42fb0a3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5r27h\" (UID: \"c6eef884-173e-4c5b-890d-3409f42fb0a3\") " pod="openshift-multus/multus-additional-cni-plugins-5r27h"
Jul 07 15:00:34.849837 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.849509    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/984f3e1f-0860-444a-bb2e-454b5d450217-iptables-alerter-script\") pod \"iptables-alerter-8n5q4\" (UID: \"984f3e1f-0860-444a-bb2e-454b5d450217\") " pod="openshift-network-operator/iptables-alerter-8n5q4"
Jul 07 15:00:34.849837 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.849533    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/984f3e1f-0860-444a-bb2e-454b5d450217-host-slash\") pod \"iptables-alerter-8n5q4\" (UID: \"984f3e1f-0860-444a-bb2e-454b5d450217\") " pod="openshift-network-operator/iptables-alerter-8n5q4"
Jul 07 15:00:34.849837 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.849556    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/671cc2c7-4b8d-4183-8a74-16d971f3431e-konnectivity-ca\") pod \"konnectivity-agent-7tg7l\" (UID: \"671cc2c7-4b8d-4183-8a74-16d971f3431e\") " pod="kube-system/konnectivity-agent-7tg7l"
Jul 07 15:00:34.849837 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.849588    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c6eef884-173e-4c5b-890d-3409f42fb0a3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5r27h\" (UID: \"c6eef884-173e-4c5b-890d-3409f42fb0a3\") " pod="openshift-multus/multus-additional-cni-plugins-5r27h"
Jul 07 15:00:34.849837 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.849618    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ebbf4a6-429c-465d-916d-84567a210db5-metrics-certs\") pod \"network-metrics-daemon-zs477\" (UID: \"0ebbf4a6-429c-465d-916d-84567a210db5\") " pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:00:34.849837 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.849635    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwxrk\" (UniqueName: \"kubernetes.io/projected/0ebbf4a6-429c-465d-916d-84567a210db5-kube-api-access-cwxrk\") pod \"network-metrics-daemon-zs477\" (UID: \"0ebbf4a6-429c-465d-916d-84567a210db5\") " pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:00:34.849837 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.849658    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/88f0b024-94cb-44b4-a5f4-9497ce96c3cb-serviceca\") pod \"node-ca-zk8hk\" (UID: \"88f0b024-94cb-44b4-a5f4-9497ce96c3cb\") " pod="openshift-image-registry/node-ca-zk8hk"
Jul 07 15:00:34.849837 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.849676    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c6eef884-173e-4c5b-890d-3409f42fb0a3-system-cni-dir\") pod \"multus-additional-cni-plugins-5r27h\" (UID: \"c6eef884-173e-4c5b-890d-3409f42fb0a3\") " pod="openshift-multus/multus-additional-cni-plugins-5r27h"
Jul 07 15:00:34.849837 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.849692    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdg6q\" (UniqueName: \"kubernetes.io/projected/c6eef884-173e-4c5b-890d-3409f42fb0a3-kube-api-access-xdg6q\") pod \"multus-additional-cni-plugins-5r27h\" (UID: \"c6eef884-173e-4c5b-890d-3409f42fb0a3\") " pod="openshift-multus/multus-additional-cni-plugins-5r27h"
Jul 07 15:00:34.849837 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.849708    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcsmp\" (UniqueName: \"kubernetes.io/projected/984f3e1f-0860-444a-bb2e-454b5d450217-kube-api-access-hcsmp\") pod \"iptables-alerter-8n5q4\" (UID: \"984f3e1f-0860-444a-bb2e-454b5d450217\") " pod="openshift-network-operator/iptables-alerter-8n5q4"
Jul 07 15:00:34.849837 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.849725    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c6eef884-173e-4c5b-890d-3409f42fb0a3-cnibin\") pod \"multus-additional-cni-plugins-5r27h\" (UID: \"c6eef884-173e-4c5b-890d-3409f42fb0a3\") " pod="openshift-multus/multus-additional-cni-plugins-5r27h"
Jul 07 15:00:34.849837 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.849762    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c6eef884-173e-4c5b-890d-3409f42fb0a3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5r27h\" (UID: \"c6eef884-173e-4c5b-890d-3409f42fb0a3\") " pod="openshift-multus/multus-additional-cni-plugins-5r27h"
Jul 07 15:00:34.849837 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.849772    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c6eef884-173e-4c5b-890d-3409f42fb0a3-cnibin\") pod \"multus-additional-cni-plugins-5r27h\" (UID: \"c6eef884-173e-4c5b-890d-3409f42fb0a3\") " pod="openshift-multus/multus-additional-cni-plugins-5r27h"
Jul 07 15:00:34.849837 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.849812    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/984f3e1f-0860-444a-bb2e-454b5d450217-host-slash\") pod \"iptables-alerter-8n5q4\" (UID: \"984f3e1f-0860-444a-bb2e-454b5d450217\") " pod="openshift-network-operator/iptables-alerter-8n5q4"
Jul 07 15:00:34.850603 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.850035    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c6eef884-173e-4c5b-890d-3409f42fb0a3-cni-binary-copy\") pod \"multus-additional-cni-plugins-5r27h\" (UID: \"c6eef884-173e-4c5b-890d-3409f42fb0a3\") " pod="openshift-multus/multus-additional-cni-plugins-5r27h"
Jul 07 15:00:34.850603 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.850199    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c6eef884-173e-4c5b-890d-3409f42fb0a3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5r27h\" (UID: \"c6eef884-173e-4c5b-890d-3409f42fb0a3\") " pod="openshift-multus/multus-additional-cni-plugins-5r27h"
Jul 07 15:00:34.850603 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.850242    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c6eef884-173e-4c5b-890d-3409f42fb0a3-system-cni-dir\") pod \"multus-additional-cni-plugins-5r27h\" (UID: \"c6eef884-173e-4c5b-890d-3409f42fb0a3\") " pod="openshift-multus/multus-additional-cni-plugins-5r27h"
Jul 07 15:00:34.850603 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.850384    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/984f3e1f-0860-444a-bb2e-454b5d450217-iptables-alerter-script\") pod \"iptables-alerter-8n5q4\" (UID: \"984f3e1f-0860-444a-bb2e-454b5d450217\") " pod="openshift-network-operator/iptables-alerter-8n5q4"
Jul 07 15:00:34.850603 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.850438    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/671cc2c7-4b8d-4183-8a74-16d971f3431e-konnectivity-ca\") pod \"konnectivity-agent-7tg7l\" (UID: \"671cc2c7-4b8d-4183-8a74-16d971f3431e\") " pod="kube-system/konnectivity-agent-7tg7l"
Jul 07 15:00:34.850603 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:34.850459    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 15:00:34.850603 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:34.850500    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ebbf4a6-429c-465d-916d-84567a210db5-metrics-certs podName:0ebbf4a6-429c-465d-916d-84567a210db5 nodeName:}" failed. No retries permitted until 2026-07-07 15:00:35.350488372 +0000 UTC m=+3.147025080 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ebbf4a6-429c-465d-916d-84567a210db5-metrics-certs") pod "network-metrics-daemon-zs477" (UID: "0ebbf4a6-429c-465d-916d-84567a210db5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 15:00:34.850603 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.850556    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c6eef884-173e-4c5b-890d-3409f42fb0a3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5r27h\" (UID: \"c6eef884-173e-4c5b-890d-3409f42fb0a3\") " pod="openshift-multus/multus-additional-cni-plugins-5r27h"
Jul 07 15:00:34.850603 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.850562    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/88f0b024-94cb-44b4-a5f4-9497ce96c3cb-serviceca\") pod \"node-ca-zk8hk\" (UID: \"88f0b024-94cb-44b4-a5f4-9497ce96c3cb\") " pod="openshift-image-registry/node-ca-zk8hk"
Jul 07 15:00:34.852525 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.852324    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/671cc2c7-4b8d-4183-8a74-16d971f3431e-agent-certs\") pod \"konnectivity-agent-7tg7l\" (UID: \"671cc2c7-4b8d-4183-8a74-16d971f3431e\") " pod="kube-system/konnectivity-agent-7tg7l"
Jul 07 15:00:34.878715 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.878685    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdg6q\" (UniqueName: \"kubernetes.io/projected/c6eef884-173e-4c5b-890d-3409f42fb0a3-kube-api-access-xdg6q\") pod \"multus-additional-cni-plugins-5r27h\" (UID: \"c6eef884-173e-4c5b-890d-3409f42fb0a3\") " pod="openshift-multus/multus-additional-cni-plugins-5r27h"
Jul 07 15:00:34.878899 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.878793    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnch2\" (UniqueName: \"kubernetes.io/projected/88f0b024-94cb-44b4-a5f4-9497ce96c3cb-kube-api-access-nnch2\") pod \"node-ca-zk8hk\" (UID: \"88f0b024-94cb-44b4-a5f4-9497ce96c3cb\") " pod="openshift-image-registry/node-ca-zk8hk"
Jul 07 15:00:34.880306 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:34.880057    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 15:00:34.880306 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:34.880079    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 15:00:34.880306 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:34.880088    2567 projected.go:196] Error preparing data for projected volume kube-api-access-jvn2q for pod openshift-network-diagnostics/network-check-target-9gmwd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 15:00:34.880306 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:34.880205    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7a85a73-907b-4a3b-bee4-b03ce0da2b6c-kube-api-access-jvn2q podName:a7a85a73-907b-4a3b-bee4-b03ce0da2b6c nodeName:}" failed. No retries permitted until 2026-07-07 15:00:35.380185039 +0000 UTC m=+3.176721750 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jvn2q" (UniqueName: "kubernetes.io/projected/a7a85a73-907b-4a3b-bee4-b03ce0da2b6c-kube-api-access-jvn2q") pod "network-check-target-9gmwd" (UID: "a7a85a73-907b-4a3b-bee4-b03ce0da2b6c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 15:00:34.882774 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.882745    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcsmp\" (UniqueName: \"kubernetes.io/projected/984f3e1f-0860-444a-bb2e-454b5d450217-kube-api-access-hcsmp\") pod \"iptables-alerter-8n5q4\" (UID: \"984f3e1f-0860-444a-bb2e-454b5d450217\") " pod="openshift-network-operator/iptables-alerter-8n5q4"
Jul 07 15:00:34.883503 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.883482    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwxrk\" (UniqueName: \"kubernetes.io/projected/0ebbf4a6-429c-465d-916d-84567a210db5-kube-api-access-cwxrk\") pod \"network-metrics-daemon-zs477\" (UID: \"0ebbf4a6-429c-465d-916d-84567a210db5\") " pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:00:34.933815 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.933777    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-d2br4"
Jul 07 15:00:34.944229 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.944204    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb"
Jul 07 15:00:34.947926 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:34.947899    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4017287a_0129_4a52_8875_045408b9a82d.slice/crio-4c4fd0d14f6506a76147f0089984c2a9351a857fad010b2441c0d01b91216c86 WatchSource:0}: Error finding container 4c4fd0d14f6506a76147f0089984c2a9351a857fad010b2441c0d01b91216c86: Status 404 returned error can't find the container with id 4c4fd0d14f6506a76147f0089984c2a9351a857fad010b2441c0d01b91216c86
Jul 07 15:00:34.956301 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.956275    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:00:34.956646 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:34.956619    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7006a3c_1f66_4a34_8ae8_915722738e79.slice/crio-9da05945bebb2ed02ece87633c48073138fdfc5b72deef14378bc2f6c585f606 WatchSource:0}: Error finding container 9da05945bebb2ed02ece87633c48073138fdfc5b72deef14378bc2f6c585f606: Status 404 returned error can't find the container with id 9da05945bebb2ed02ece87633c48073138fdfc5b72deef14378bc2f6c585f606
Jul 07 15:00:34.964231 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.964210    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-z8276"
Jul 07 15:00:34.966103 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:34.966070    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca0a1db1_0575_4ea6_a93a_6445a39114b2.slice/crio-5c28f553ff1f03485f5ca2ef2e6114a259345e929b4c2eaaa498afd714ac008c WatchSource:0}: Error finding container 5c28f553ff1f03485f5ca2ef2e6114a259345e929b4c2eaaa498afd714ac008c: Status 404 returned error can't find the container with id 5c28f553ff1f03485f5ca2ef2e6114a259345e929b4c2eaaa498afd714ac008c
Jul 07 15:00:34.971933 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.971907    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vgrxt"
Jul 07 15:00:34.974346 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:34.974315    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b0751af_98d6_4f05_ae7f_d1cff06cf817.slice/crio-790117cbdd9b32e4675a1b1f570e68d7022393a7dc834004afefd6f02b1556b8 WatchSource:0}: Error finding container 790117cbdd9b32e4675a1b1f570e68d7022393a7dc834004afefd6f02b1556b8: Status 404 returned error can't find the container with id 790117cbdd9b32e4675a1b1f570e68d7022393a7dc834004afefd6f02b1556b8
Jul 07 15:00:34.979654 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.979631    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5r27h"
Jul 07 15:00:34.981614 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:34.981592    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb18f878a_7e28_4cb3_92b9_521847f10288.slice/crio-b71fc5e560533782c662bf768589a09bf7f7edd01b82cecdf2a73ce6e616205b WatchSource:0}: Error finding container b71fc5e560533782c662bf768589a09bf7f7edd01b82cecdf2a73ce6e616205b: Status 404 returned error can't find the container with id b71fc5e560533782c662bf768589a09bf7f7edd01b82cecdf2a73ce6e616205b
Jul 07 15:00:34.989468 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.989447    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8n5q4"
Jul 07 15:00:34.991172 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:34.991145    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6eef884_173e_4c5b_890d_3409f42fb0a3.slice/crio-ea3800a9bd7780ede19b2a224414d68163e7fba2339278edab19b990dbfb0c70 WatchSource:0}: Error finding container ea3800a9bd7780ede19b2a224414d68163e7fba2339278edab19b990dbfb0c70: Status 404 returned error can't find the container with id ea3800a9bd7780ede19b2a224414d68163e7fba2339278edab19b990dbfb0c70
Jul 07 15:00:34.995431 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:34.995363    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7tg7l"
Jul 07 15:00:34.999548 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:34.999524    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod984f3e1f_0860_444a_bb2e_454b5d450217.slice/crio-4c3bc2ee25bbf6ddff0a68faff37123ca358ce0af17c629875e8629f8b69db74 WatchSource:0}: Error finding container 4c3bc2ee25bbf6ddff0a68faff37123ca358ce0af17c629875e8629f8b69db74: Status 404 returned error can't find the container with id 4c3bc2ee25bbf6ddff0a68faff37123ca358ce0af17c629875e8629f8b69db74
Jul 07 15:00:35.001037 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:35.001022    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zk8hk"
Jul 07 15:00:35.253061 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:35.252972    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-original-pull-secret\") pod \"global-pull-secret-syncer-zz5ql\" (UID: \"31b4f468-dc43-4504-b8c1-3f6b91c08bc5\") " pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:00:35.253207 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:35.253129    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 15:00:35.253259 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:35.253209    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-original-pull-secret podName:31b4f468-dc43-4504-b8c1-3f6b91c08bc5 nodeName:}" failed. No retries permitted until 2026-07-07 15:00:36.253190788 +0000 UTC m=+4.049727520 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-original-pull-secret") pod "global-pull-secret-syncer-zz5ql" (UID: "31b4f468-dc43-4504-b8c1-3f6b91c08bc5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 15:00:35.353906 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:35.353862    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ebbf4a6-429c-465d-916d-84567a210db5-metrics-certs\") pod \"network-metrics-daemon-zs477\" (UID: \"0ebbf4a6-429c-465d-916d-84567a210db5\") " pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:00:35.354077 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:35.354054    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 15:00:35.354148 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:35.354133    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ebbf4a6-429c-465d-916d-84567a210db5-metrics-certs podName:0ebbf4a6-429c-465d-916d-84567a210db5 nodeName:}" failed. No retries permitted until 2026-07-07 15:00:36.354112208 +0000 UTC m=+4.150648934 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ebbf4a6-429c-465d-916d-84567a210db5-metrics-certs") pod "network-metrics-daemon-zs477" (UID: "0ebbf4a6-429c-465d-916d-84567a210db5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 15:00:35.454845 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:35.454773    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvn2q\" (UniqueName: \"kubernetes.io/projected/a7a85a73-907b-4a3b-bee4-b03ce0da2b6c-kube-api-access-jvn2q\") pod \"network-check-target-9gmwd\" (UID: \"a7a85a73-907b-4a3b-bee4-b03ce0da2b6c\") " pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:00:35.455069 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:35.454965    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 15:00:35.455069 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:35.454985    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 15:00:35.455069 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:35.454998    2567 projected.go:196] Error preparing data for projected volume kube-api-access-jvn2q for pod openshift-network-diagnostics/network-check-target-9gmwd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 15:00:35.455069 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:35.455067    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7a85a73-907b-4a3b-bee4-b03ce0da2b6c-kube-api-access-jvn2q podName:a7a85a73-907b-4a3b-bee4-b03ce0da2b6c nodeName:}" failed. No retries permitted until 2026-07-07 15:00:36.455048645 +0000 UTC m=+4.251585354 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jvn2q" (UniqueName: "kubernetes.io/projected/a7a85a73-907b-4a3b-bee4-b03ce0da2b6c-kube-api-access-jvn2q") pod "network-check-target-9gmwd" (UID: "a7a85a73-907b-4a3b-bee4-b03ce0da2b6c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 15:00:35.772268 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:35.771742    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:00:35.772268 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:35.771918    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zz5ql" podUID="31b4f468-dc43-4504-b8c1-3f6b91c08bc5"
Jul 07 15:00:35.788471 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:35.788106    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 14:55:35 +0000 UTC" deadline="2028-02-09 06:37:43.325306108 +0000 UTC"
Jul 07 15:00:35.788471 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:35.788141    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13959h37m7.537169255s"
Jul 07 15:00:35.788471 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:35.788378    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb" event={"ID":"c7006a3c-1f66-4a34-8ae8-915722738e79","Type":"ContainerStarted","Data":"9da05945bebb2ed02ece87633c48073138fdfc5b72deef14378bc2f6c585f606"}
Jul 07 15:00:35.790181 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:35.790102    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-z8276" event={"ID":"8b0751af-98d6-4f05-ae7f-d1cff06cf817","Type":"ContainerStarted","Data":"790117cbdd9b32e4675a1b1f570e68d7022393a7dc834004afefd6f02b1556b8"}
Jul 07 15:00:35.791470 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:35.791401    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-d2br4" event={"ID":"4017287a-0129-4a52-8875-045408b9a82d","Type":"ContainerStarted","Data":"4c4fd0d14f6506a76147f0089984c2a9351a857fad010b2441c0d01b91216c86"}
Jul 07 15:00:35.792949 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:35.792859    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8n5q4" event={"ID":"984f3e1f-0860-444a-bb2e-454b5d450217","Type":"ContainerStarted","Data":"4c3bc2ee25bbf6ddff0a68faff37123ca358ce0af17c629875e8629f8b69db74"}
Jul 07 15:00:35.794215 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:35.794152    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5r27h" event={"ID":"c6eef884-173e-4c5b-890d-3409f42fb0a3","Type":"ContainerStarted","Data":"ea3800a9bd7780ede19b2a224414d68163e7fba2339278edab19b990dbfb0c70"}
Jul 07 15:00:35.795457 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:35.795393    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vgrxt" event={"ID":"b18f878a-7e28-4cb3-92b9-521847f10288","Type":"ContainerStarted","Data":"b71fc5e560533782c662bf768589a09bf7f7edd01b82cecdf2a73ce6e616205b"}
Jul 07 15:00:35.797059 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:35.796958    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xj52l" event={"ID":"ca0a1db1-0575-4ea6-a93a-6445a39114b2","Type":"ContainerStarted","Data":"5c28f553ff1f03485f5ca2ef2e6114a259345e929b4c2eaaa498afd714ac008c"}
Jul 07 15:00:35.868943 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:35.868892    2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 15:00:36.262199 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:36.262159    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-original-pull-secret\") pod \"global-pull-secret-syncer-zz5ql\" (UID: \"31b4f468-dc43-4504-b8c1-3f6b91c08bc5\") " pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:00:36.262365 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:36.262334    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 15:00:36.262448 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:36.262405    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-original-pull-secret podName:31b4f468-dc43-4504-b8c1-3f6b91c08bc5 nodeName:}" failed. No retries permitted until 2026-07-07 15:00:38.262386315 +0000 UTC m=+6.058923024 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-original-pull-secret") pod "global-pull-secret-syncer-zz5ql" (UID: "31b4f468-dc43-4504-b8c1-3f6b91c08bc5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 15:00:36.365079 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:36.365028    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ebbf4a6-429c-465d-916d-84567a210db5-metrics-certs\") pod \"network-metrics-daemon-zs477\" (UID: \"0ebbf4a6-429c-465d-916d-84567a210db5\") " pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:00:36.365277 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:36.365247    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 15:00:36.365358 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:36.365338    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ebbf4a6-429c-465d-916d-84567a210db5-metrics-certs podName:0ebbf4a6-429c-465d-916d-84567a210db5 nodeName:}" failed. No retries permitted until 2026-07-07 15:00:38.365317049 +0000 UTC m=+6.161853777 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ebbf4a6-429c-465d-916d-84567a210db5-metrics-certs") pod "network-metrics-daemon-zs477" (UID: "0ebbf4a6-429c-465d-916d-84567a210db5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 15:00:36.466079 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:36.466036    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvn2q\" (UniqueName: \"kubernetes.io/projected/a7a85a73-907b-4a3b-bee4-b03ce0da2b6c-kube-api-access-jvn2q\") pod \"network-check-target-9gmwd\" (UID: \"a7a85a73-907b-4a3b-bee4-b03ce0da2b6c\") " pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:00:36.466280 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:36.466241    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 15:00:36.466280 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:36.466277    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 15:00:36.466423 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:36.466293    2567 projected.go:196] Error preparing data for projected volume kube-api-access-jvn2q for pod openshift-network-diagnostics/network-check-target-9gmwd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 15:00:36.466423 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:36.466358    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7a85a73-907b-4a3b-bee4-b03ce0da2b6c-kube-api-access-jvn2q podName:a7a85a73-907b-4a3b-bee4-b03ce0da2b6c nodeName:}" failed. No retries permitted until 2026-07-07 15:00:38.466337131 +0000 UTC m=+6.262873854 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jvn2q" (UniqueName: "kubernetes.io/projected/a7a85a73-907b-4a3b-bee4-b03ce0da2b6c-kube-api-access-jvn2q") pod "network-check-target-9gmwd" (UID: "a7a85a73-907b-4a3b-bee4-b03ce0da2b6c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 15:00:36.494755 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:36.494710    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod671cc2c7_4b8d_4183_8a74_16d971f3431e.slice/crio-10ffde83cb03b4f5b1db68d4e3a60d8ce9507cd53495b341dba871c9463cb5ef WatchSource:0}: Error finding container 10ffde83cb03b4f5b1db68d4e3a60d8ce9507cd53495b341dba871c9463cb5ef: Status 404 returned error can't find the container with id 10ffde83cb03b4f5b1db68d4e3a60d8ce9507cd53495b341dba871c9463cb5ef
Jul 07 15:00:36.500819 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:00:36.500790    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88f0b024_94cb_44b4_a5f4_9497ce96c3cb.slice/crio-2b00dbb997c15c833a3059740b51a49f3b88515c02a382a306cfb07f992575f8 WatchSource:0}: Error finding container 2b00dbb997c15c833a3059740b51a49f3b88515c02a382a306cfb07f992575f8: Status 404 returned error can't find the container with id 2b00dbb997c15c833a3059740b51a49f3b88515c02a382a306cfb07f992575f8
Jul 07 15:00:36.776343 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:36.776033    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:00:36.776810 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:36.776383    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gmwd" podUID="a7a85a73-907b-4a3b-bee4-b03ce0da2b6c"
Jul 07 15:00:36.776810 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:36.776486    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:00:36.776810 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:36.776575    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zs477" podUID="0ebbf4a6-429c-465d-916d-84567a210db5"
Jul 07 15:00:36.788988 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:36.788925    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 14:55:35 +0000 UTC" deadline="2028-03-12 14:11:25.122725646 +0000 UTC"
Jul 07 15:00:36.788988 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:36.788986    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14735h10m48.3337442s"
Jul 07 15:00:36.818963 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:36.818862    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-233.ec2.internal" event={"ID":"938243ce709c303fec6b51c38490eae8","Type":"ContainerStarted","Data":"66cb9b426409f259717e65f9002e47870619c241f625c2a40fb29bb0210dce53"}
Jul 07 15:00:36.828386 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:36.828292    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zk8hk" event={"ID":"88f0b024-94cb-44b4-a5f4-9497ce96c3cb","Type":"ContainerStarted","Data":"2b00dbb997c15c833a3059740b51a49f3b88515c02a382a306cfb07f992575f8"}
Jul 07 15:00:36.836901 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:36.836818    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7tg7l" event={"ID":"671cc2c7-4b8d-4183-8a74-16d971f3431e","Type":"ContainerStarted","Data":"10ffde83cb03b4f5b1db68d4e3a60d8ce9507cd53495b341dba871c9463cb5ef"}
Jul 07 15:00:37.771595 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:37.771564    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:00:37.771781 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:37.771698    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zz5ql" podUID="31b4f468-dc43-4504-b8c1-3f6b91c08bc5"
Jul 07 15:00:37.854941 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:37.853966    2567 generic.go:356] "Generic (PLEG): container finished" podID="9a9307618ad481c92addb0219af76bb7" containerID="5be2d0058822c2d76986048c53f8704fa3fef3149c5824fbfb38babb934fd5b8" exitCode=0
Jul 07 15:00:37.855968 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:37.855563    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal" event={"ID":"9a9307618ad481c92addb0219af76bb7","Type":"ContainerDied","Data":"5be2d0058822c2d76986048c53f8704fa3fef3149c5824fbfb38babb934fd5b8"}
Jul 07 15:00:37.881885 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:37.881116    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-233.ec2.internal" podStartSLOduration=3.881094124 podStartE2EDuration="3.881094124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:00:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:00:36.839433453 +0000 UTC m=+4.635970188" watchObservedRunningTime="2026-07-07 15:00:37.881094124 +0000 UTC m=+5.677630858"
Jul 07 15:00:38.287719 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:38.287075    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-original-pull-secret\") pod \"global-pull-secret-syncer-zz5ql\" (UID: \"31b4f468-dc43-4504-b8c1-3f6b91c08bc5\") " pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:00:38.287719 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:38.287287    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 15:00:38.287719 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:38.287358    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-original-pull-secret podName:31b4f468-dc43-4504-b8c1-3f6b91c08bc5 nodeName:}" failed. No retries permitted until 2026-07-07 15:00:42.28733692 +0000 UTC m=+10.083873631 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-original-pull-secret") pod "global-pull-secret-syncer-zz5ql" (UID: "31b4f468-dc43-4504-b8c1-3f6b91c08bc5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 15:00:38.388948 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:38.388256    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ebbf4a6-429c-465d-916d-84567a210db5-metrics-certs\") pod \"network-metrics-daemon-zs477\" (UID: \"0ebbf4a6-429c-465d-916d-84567a210db5\") " pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:00:38.388948 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:38.388419    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 15:00:38.388948 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:38.388490    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ebbf4a6-429c-465d-916d-84567a210db5-metrics-certs podName:0ebbf4a6-429c-465d-916d-84567a210db5 nodeName:}" failed. No retries permitted until 2026-07-07 15:00:42.388469703 +0000 UTC m=+10.185006414 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ebbf4a6-429c-465d-916d-84567a210db5-metrics-certs") pod "network-metrics-daemon-zs477" (UID: "0ebbf4a6-429c-465d-916d-84567a210db5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 15:00:38.489334 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:38.489287    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvn2q\" (UniqueName: \"kubernetes.io/projected/a7a85a73-907b-4a3b-bee4-b03ce0da2b6c-kube-api-access-jvn2q\") pod \"network-check-target-9gmwd\" (UID: \"a7a85a73-907b-4a3b-bee4-b03ce0da2b6c\") " pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:00:38.489537 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:38.489471    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 15:00:38.489537 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:38.489493    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 15:00:38.489537 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:38.489505    2567 projected.go:196] Error preparing data for projected volume kube-api-access-jvn2q for pod openshift-network-diagnostics/network-check-target-9gmwd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 15:00:38.489686 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:38.489565    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7a85a73-907b-4a3b-bee4-b03ce0da2b6c-kube-api-access-jvn2q podName:a7a85a73-907b-4a3b-bee4-b03ce0da2b6c nodeName:}" failed. No retries permitted until 2026-07-07 15:00:42.489546958 +0000 UTC m=+10.286083687 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jvn2q" (UniqueName: "kubernetes.io/projected/a7a85a73-907b-4a3b-bee4-b03ce0da2b6c-kube-api-access-jvn2q") pod "network-check-target-9gmwd" (UID: "a7a85a73-907b-4a3b-bee4-b03ce0da2b6c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 15:00:38.772414 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:38.772320    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:00:38.772414 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:38.772352    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:00:38.772647 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:38.772473    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zs477" podUID="0ebbf4a6-429c-465d-916d-84567a210db5"
Jul 07 15:00:38.772647 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:38.772589    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gmwd" podUID="a7a85a73-907b-4a3b-bee4-b03ce0da2b6c"
Jul 07 15:00:38.872940 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:38.872794    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal" event={"ID":"9a9307618ad481c92addb0219af76bb7","Type":"ContainerStarted","Data":"1ecf136fd9281f1ea48ed4c73934393581ea018c7ed531d2dcf1401c711c45eb"}
Jul 07 15:00:38.895837 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:38.895557    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-233.ec2.internal" podStartSLOduration=4.895531308 podStartE2EDuration="4.895531308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:00:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:00:38.894335558 +0000 UTC m=+6.690872320" watchObservedRunningTime="2026-07-07 15:00:38.895531308 +0000 UTC m=+6.692068040"
Jul 07 15:00:39.772335 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:39.772296    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:00:39.772543 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:39.772434    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zz5ql" podUID="31b4f468-dc43-4504-b8c1-3f6b91c08bc5"
Jul 07 15:00:40.772051 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:40.772019    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:00:40.772479 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:40.772166    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gmwd" podUID="a7a85a73-907b-4a3b-bee4-b03ce0da2b6c"
Jul 07 15:00:40.772850 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:40.772595    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:00:40.772850 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:40.772717    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zs477" podUID="0ebbf4a6-429c-465d-916d-84567a210db5"
Jul 07 15:00:41.772428 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:41.772390    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:00:41.772938 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:41.772646    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zz5ql" podUID="31b4f468-dc43-4504-b8c1-3f6b91c08bc5"
Jul 07 15:00:42.324741 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:42.324033    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-original-pull-secret\") pod \"global-pull-secret-syncer-zz5ql\" (UID: \"31b4f468-dc43-4504-b8c1-3f6b91c08bc5\") " pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:00:42.324741 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:42.324244    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 15:00:42.324741 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:42.324310    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-original-pull-secret podName:31b4f468-dc43-4504-b8c1-3f6b91c08bc5 nodeName:}" failed. No retries permitted until 2026-07-07 15:00:50.324290473 +0000 UTC m=+18.120827208 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-original-pull-secret") pod "global-pull-secret-syncer-zz5ql" (UID: "31b4f468-dc43-4504-b8c1-3f6b91c08bc5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 15:00:42.425529 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:42.424909    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ebbf4a6-429c-465d-916d-84567a210db5-metrics-certs\") pod \"network-metrics-daemon-zs477\" (UID: \"0ebbf4a6-429c-465d-916d-84567a210db5\") " pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:00:42.425529 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:42.425084    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 15:00:42.425529 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:42.425150    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ebbf4a6-429c-465d-916d-84567a210db5-metrics-certs podName:0ebbf4a6-429c-465d-916d-84567a210db5 nodeName:}" failed. No retries permitted until 2026-07-07 15:00:50.42513055 +0000 UTC m=+18.221667273 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ebbf4a6-429c-465d-916d-84567a210db5-metrics-certs") pod "network-metrics-daemon-zs477" (UID: "0ebbf4a6-429c-465d-916d-84567a210db5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 15:00:42.525858 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:42.525817    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvn2q\" (UniqueName: \"kubernetes.io/projected/a7a85a73-907b-4a3b-bee4-b03ce0da2b6c-kube-api-access-jvn2q\") pod \"network-check-target-9gmwd\" (UID: \"a7a85a73-907b-4a3b-bee4-b03ce0da2b6c\") " pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:00:42.526075 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:42.526046    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 15:00:42.526075 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:42.526079    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 15:00:42.526265 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:42.526094    2567 projected.go:196] Error preparing data for projected volume kube-api-access-jvn2q for pod openshift-network-diagnostics/network-check-target-9gmwd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 15:00:42.526265 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:42.526171    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7a85a73-907b-4a3b-bee4-b03ce0da2b6c-kube-api-access-jvn2q podName:a7a85a73-907b-4a3b-bee4-b03ce0da2b6c nodeName:}" failed. No retries permitted until 2026-07-07 15:00:50.526149155 +0000 UTC m=+18.322685886 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jvn2q" (UniqueName: "kubernetes.io/projected/a7a85a73-907b-4a3b-bee4-b03ce0da2b6c-kube-api-access-jvn2q") pod "network-check-target-9gmwd" (UID: "a7a85a73-907b-4a3b-bee4-b03ce0da2b6c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 15:00:42.773265 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:42.773186    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:00:42.773783 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:42.773322    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zs477" podUID="0ebbf4a6-429c-465d-916d-84567a210db5"
Jul 07 15:00:42.773783 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:42.773593    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:00:42.773783 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:42.773658    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gmwd" podUID="a7a85a73-907b-4a3b-bee4-b03ce0da2b6c"
Jul 07 15:00:43.771461 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:43.771429    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:00:43.771712 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:43.771537    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zz5ql" podUID="31b4f468-dc43-4504-b8c1-3f6b91c08bc5"
Jul 07 15:00:44.772446 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:44.771930    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:00:44.772446 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:44.772078    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zs477" podUID="0ebbf4a6-429c-465d-916d-84567a210db5"
Jul 07 15:00:44.772944 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:44.772565    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:00:44.772944 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:44.772669    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gmwd" podUID="a7a85a73-907b-4a3b-bee4-b03ce0da2b6c"
Jul 07 15:00:45.772752 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:45.772294    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:00:45.772752 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:45.772418    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zz5ql" podUID="31b4f468-dc43-4504-b8c1-3f6b91c08bc5"
Jul 07 15:00:46.772162 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:46.772029    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:00:46.772162 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:46.772080    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:00:46.772162 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:46.772156    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zs477" podUID="0ebbf4a6-429c-465d-916d-84567a210db5"
Jul 07 15:00:46.772409 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:46.772241    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gmwd" podUID="a7a85a73-907b-4a3b-bee4-b03ce0da2b6c"
Jul 07 15:00:47.772092 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:47.772057    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:00:47.772574 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:47.772203    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zz5ql" podUID="31b4f468-dc43-4504-b8c1-3f6b91c08bc5"
Jul 07 15:00:48.773709 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:48.773676    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:00:48.774152 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:48.773678    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:00:48.774152 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:48.773818    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zs477" podUID="0ebbf4a6-429c-465d-916d-84567a210db5"
Jul 07 15:00:48.774152 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:48.773925    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gmwd" podUID="a7a85a73-907b-4a3b-bee4-b03ce0da2b6c"
Jul 07 15:00:49.771595 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:49.771556    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:00:49.771860 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:49.771690    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zz5ql" podUID="31b4f468-dc43-4504-b8c1-3f6b91c08bc5"
Jul 07 15:00:50.387604 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:50.387559    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-original-pull-secret\") pod \"global-pull-secret-syncer-zz5ql\" (UID: \"31b4f468-dc43-4504-b8c1-3f6b91c08bc5\") " pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:00:50.388102 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:50.387724    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 15:00:50.388102 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:50.387801    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-original-pull-secret podName:31b4f468-dc43-4504-b8c1-3f6b91c08bc5 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:06.387780712 +0000 UTC m=+34.184317442 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-original-pull-secret") pod "global-pull-secret-syncer-zz5ql" (UID: "31b4f468-dc43-4504-b8c1-3f6b91c08bc5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 15:00:50.488807 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:50.488767    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ebbf4a6-429c-465d-916d-84567a210db5-metrics-certs\") pod \"network-metrics-daemon-zs477\" (UID: \"0ebbf4a6-429c-465d-916d-84567a210db5\") " pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:00:50.488999 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:50.488938    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 15:00:50.488999 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:50.489000    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ebbf4a6-429c-465d-916d-84567a210db5-metrics-certs podName:0ebbf4a6-429c-465d-916d-84567a210db5 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:06.488985946 +0000 UTC m=+34.285522652 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ebbf4a6-429c-465d-916d-84567a210db5-metrics-certs") pod "network-metrics-daemon-zs477" (UID: "0ebbf4a6-429c-465d-916d-84567a210db5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 15:00:50.589911 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:50.589864    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvn2q\" (UniqueName: \"kubernetes.io/projected/a7a85a73-907b-4a3b-bee4-b03ce0da2b6c-kube-api-access-jvn2q\") pod \"network-check-target-9gmwd\" (UID: \"a7a85a73-907b-4a3b-bee4-b03ce0da2b6c\") " pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:00:50.590094 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:50.590062    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 15:00:50.590094 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:50.590088    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 15:00:50.590199 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:50.590102    2567 projected.go:196] Error preparing data for projected volume kube-api-access-jvn2q for pod openshift-network-diagnostics/network-check-target-9gmwd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 15:00:50.590199 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:50.590168    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7a85a73-907b-4a3b-bee4-b03ce0da2b6c-kube-api-access-jvn2q podName:a7a85a73-907b-4a3b-bee4-b03ce0da2b6c nodeName:}" failed. No retries permitted until 2026-07-07 15:01:06.590149734 +0000 UTC m=+34.386686443 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jvn2q" (UniqueName: "kubernetes.io/projected/a7a85a73-907b-4a3b-bee4-b03ce0da2b6c-kube-api-access-jvn2q") pod "network-check-target-9gmwd" (UID: "a7a85a73-907b-4a3b-bee4-b03ce0da2b6c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 15:00:50.771991 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:50.771912    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:00:50.771991 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:50.771938    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:00:50.772205 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:50.772047    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zs477" podUID="0ebbf4a6-429c-465d-916d-84567a210db5"
Jul 07 15:00:50.772205 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:50.772155    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gmwd" podUID="a7a85a73-907b-4a3b-bee4-b03ce0da2b6c"
Jul 07 15:00:51.771486 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:51.771449    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:00:51.771909 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:51.771589    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zz5ql" podUID="31b4f468-dc43-4504-b8c1-3f6b91c08bc5"
Jul 07 15:00:52.772027 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:52.771989    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:00:52.772456 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:52.772094    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gmwd" podUID="a7a85a73-907b-4a3b-bee4-b03ce0da2b6c"
Jul 07 15:00:52.772456 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:52.772152    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:00:52.772456 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:52.772281    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zs477" podUID="0ebbf4a6-429c-465d-916d-84567a210db5"
Jul 07 15:00:53.772109 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:53.772072    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:00:53.772543 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:53.772207    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zz5ql" podUID="31b4f468-dc43-4504-b8c1-3f6b91c08bc5"
Jul 07 15:00:54.772050 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:54.771561    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:00:54.772050 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:54.771681    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gmwd" podUID="a7a85a73-907b-4a3b-bee4-b03ce0da2b6c"
Jul 07 15:00:54.772050 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:54.771742    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:00:54.772050 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:54.771901    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zs477" podUID="0ebbf4a6-429c-465d-916d-84567a210db5"
Jul 07 15:00:55.771951 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:55.771756    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:00:55.772164 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:55.772033    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zz5ql" podUID="31b4f468-dc43-4504-b8c1-3f6b91c08bc5"
Jul 07 15:00:55.907121 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:55.907093    2567 generic.go:356] "Generic (PLEG): container finished" podID="c6eef884-173e-4c5b-890d-3409f42fb0a3" containerID="9c09353169ebd956d1ed44dd73a4c86ca565d0c40ac058da7a9865113b1f7bae" exitCode=0
Jul 07 15:00:55.907698 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:55.907176    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5r27h" event={"ID":"c6eef884-173e-4c5b-890d-3409f42fb0a3","Type":"ContainerDied","Data":"9c09353169ebd956d1ed44dd73a4c86ca565d0c40ac058da7a9865113b1f7bae"}
Jul 07 15:00:55.908679 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:55.908613    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vgrxt" event={"ID":"b18f878a-7e28-4cb3-92b9-521847f10288","Type":"ContainerStarted","Data":"0416ee04c08eb072e2691cf89fc576b6f9ea9979c016a6be5849bffd8863ba7c"}
Jul 07 15:00:55.911177 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:55.911146    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xj52l" event={"ID":"ca0a1db1-0575-4ea6-a93a-6445a39114b2","Type":"ContainerStarted","Data":"99bf9690684bf8dbe4fd52afe44d5d68e1bfad65482537931f9ce650f33c8d98"}
Jul 07 15:00:55.911280 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:55.911224    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xj52l" event={"ID":"ca0a1db1-0575-4ea6-a93a-6445a39114b2","Type":"ContainerStarted","Data":"d22413a2f2bf90b296dfe21b106738cd09a9b4f0238d235c5b1d319be6ca4e2e"}
Jul 07 15:00:55.911280 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:55.911270    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xj52l" event={"ID":"ca0a1db1-0575-4ea6-a93a-6445a39114b2","Type":"ContainerStarted","Data":"104fdb9e41f3a4b705a431478c985c5027d4930c4868d1e93704df66452b06cb"}
Jul 07 15:00:55.911390 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:55.911304    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xj52l" event={"ID":"ca0a1db1-0575-4ea6-a93a-6445a39114b2","Type":"ContainerStarted","Data":"b1ec89a5a5daed9d4d00a2a47ef5f20c112bba24247d966b4d2d582c6181f3cd"}
Jul 07 15:00:55.912490 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:55.912463    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb" event={"ID":"c7006a3c-1f66-4a34-8ae8-915722738e79","Type":"ContainerStarted","Data":"40d70077d1affde095d4e536a5d24dfe08d6559824f238d3ec1c2cd135d66cff"}
Jul 07 15:00:55.913956 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:55.913936    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zk8hk" event={"ID":"88f0b024-94cb-44b4-a5f4-9497ce96c3cb","Type":"ContainerStarted","Data":"f07aa2e07c2a229295e70a12ef98143b3dac9e97d64465c4e9586a105cab8cd7"}
Jul 07 15:00:55.916300 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:55.916271    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7tg7l" event={"ID":"671cc2c7-4b8d-4183-8a74-16d971f3431e","Type":"ContainerStarted","Data":"dc127b82c8f8027997b16e1694e53cf891eae1d99c730d608183a97a83448c91"}
Jul 07 15:00:55.917782 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:55.917752    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-z8276" event={"ID":"8b0751af-98d6-4f05-ae7f-d1cff06cf817","Type":"ContainerStarted","Data":"3f4c52823ab7a6f0967ba836a5161344d341d27c6cc8a2bc92cc4b92da70df4b"}
Jul 07 15:00:55.919323 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:55.919294    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-d2br4" event={"ID":"4017287a-0129-4a52-8875-045408b9a82d","Type":"ContainerStarted","Data":"f5089ec0c887a7f346b2c26bcf01120c9524c99d4c61fb640641544015e65615"}
Jul 07 15:00:55.950488 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:55.950448    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7tg7l" podStartSLOduration=4.09588691 podStartE2EDuration="22.950431456s" totalImagesPullingTime="18.854544546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:00:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:00:55.950246277 +0000 UTC m=+23.746783004" watchObservedRunningTime="2026-07-07 15:00:55.950431456 +0000 UTC m=+23.746968183"
Jul 07 15:00:55.966787 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:55.966739    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zk8hk" podStartSLOduration=4.117641275 podStartE2EDuration="22.966726746s" totalImagesPullingTime="18.849085471s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:00:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:00:55.966333559 +0000 UTC m=+23.762870287" watchObservedRunningTime="2026-07-07 15:00:55.966726746 +0000 UTC m=+23.763263489"
Jul 07 15:00:55.986913 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:55.986865    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-d2br4" podStartSLOduration=2.580916943 podStartE2EDuration="22.986823439s" totalImagesPullingTime="20.405906496s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:00:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:00:55.986782383 +0000 UTC m=+23.783319111" watchObservedRunningTime="2026-07-07 15:00:55.986823439 +0000 UTC m=+23.783360167"
Jul 07 15:00:56.010465 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:56.010421    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-z8276" podStartSLOduration=2.5384424880000003 podStartE2EDuration="23.010407384s" totalImagesPullingTime="20.471964896s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:00:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:00:56.009851962 +0000 UTC m=+23.806388781" watchObservedRunningTime="2026-07-07 15:00:56.010407384 +0000 UTC m=+23.806944111"
Jul 07 15:00:56.028716 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:56.028608    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vgrxt" podStartSLOduration=2.624653232 podStartE2EDuration="23.028590998s" totalImagesPullingTime="20.403937766s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:00:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:00:56.028300618 +0000 UTC m=+23.824837347" watchObservedRunningTime="2026-07-07 15:00:56.028590998 +0000 UTC m=+23.825127727"
Jul 07 15:00:56.772385 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:56.772354    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:00:56.772554 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:56.772365    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:00:56.772554 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:56.772460    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zs477" podUID="0ebbf4a6-429c-465d-916d-84567a210db5"
Jul 07 15:00:56.772554 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:56.772519    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gmwd" podUID="a7a85a73-907b-4a3b-bee4-b03ce0da2b6c"
Jul 07 15:00:56.924158 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:56.923869    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8n5q4" event={"ID":"984f3e1f-0860-444a-bb2e-454b5d450217","Type":"ContainerStarted","Data":"56d7471c6b8584314ed81705ef5f264a369f54f7393b8e5653e1a1efb6539963"}
Jul 07 15:00:56.928273 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:56.927812    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xj52l" event={"ID":"ca0a1db1-0575-4ea6-a93a-6445a39114b2","Type":"ContainerStarted","Data":"55e8d373c81352004b9a910e3bf9ae1217d52b9c81c133140f12ffe8f5005d22"}
Jul 07 15:00:56.928273 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:56.927910    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xj52l" event={"ID":"ca0a1db1-0575-4ea6-a93a-6445a39114b2","Type":"ContainerStarted","Data":"5dbfa51e8d9c68e0ec65dcdb082d77bc32f76af3522e56aefcebee1d55b11680"}
Jul 07 15:00:56.941917 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:56.941861    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8n5q4" podStartSLOduration=3.589367056 podStartE2EDuration="23.941820009s" totalImagesPullingTime="20.352452953s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:00:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:00:56.941488537 +0000 UTC m=+24.738025293" watchObservedRunningTime="2026-07-07 15:00:56.941820009 +0000 UTC m=+24.738356736"
Jul 07 15:00:57.312804 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:57.312768    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7tg7l"
Jul 07 15:00:57.313397 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:57.313362    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7tg7l"
Jul 07 15:00:57.571890 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:57.571813    2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 07 15:00:57.702980 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:57.702883    2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T15:00:57.571852103Z","UUID":"9d2c156f-0048-4ce8-a560-c34d5eefe605","Handler":null,"Name":"","Endpoint":""}
Jul 07 15:00:57.704230 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:57.704212    2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 07 15:00:57.704321 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:57.704247    2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 07 15:00:57.772331 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:57.772293    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:00:57.772531 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:57.772480    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zz5ql" podUID="31b4f468-dc43-4504-b8c1-3f6b91c08bc5"
Jul 07 15:00:57.931985 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:57.931892    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb" event={"ID":"c7006a3c-1f66-4a34-8ae8-915722738e79","Type":"ContainerStarted","Data":"68af61b477cc2d29349d1421622b1ab2cf118a17ea7b14903dd54faeec0d5698"}
Jul 07 15:00:57.932363 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:57.932120    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7tg7l"
Jul 07 15:00:57.932616 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:57.932599    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7tg7l"
Jul 07 15:00:58.771428 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:58.771394    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:00:58.771617 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:58.771396    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:00:58.771617 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:58.771539    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gmwd" podUID="a7a85a73-907b-4a3b-bee4-b03ce0da2b6c"
Jul 07 15:00:58.771617 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:58.771588    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zs477" podUID="0ebbf4a6-429c-465d-916d-84567a210db5"
Jul 07 15:00:58.936638 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:58.936606    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xj52l" event={"ID":"ca0a1db1-0575-4ea6-a93a-6445a39114b2","Type":"ContainerStarted","Data":"16fd9893d2001db0d48423120ee3ec0169ad771e76902a11ad465e7ee6e6abb5"}
Jul 07 15:00:59.772288 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:59.772253    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:00:59.772487 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:00:59.772368    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zz5ql" podUID="31b4f468-dc43-4504-b8c1-3f6b91c08bc5"
Jul 07 15:00:59.941438 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:59.941399    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb" event={"ID":"c7006a3c-1f66-4a34-8ae8-915722738e79","Type":"ContainerStarted","Data":"bf580adb1cb1873996fb1bcb5e1e7e5880283f0fa7e8c558b6e0d151aadb67ff"}
Jul 07 15:00:59.967356 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:00:59.967309    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbwtb" podStartSLOduration=3.321578497 podStartE2EDuration="26.967295904s" totalImagesPullingTime="23.645717407s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:00:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:00:59.965936467 +0000 UTC m=+27.762473215" watchObservedRunningTime="2026-07-07 15:00:59.967295904 +0000 UTC m=+27.763832632"
Jul 07 15:01:00.771376 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:00.771297    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:01:00.771376 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:00.771340    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:01:00.771562 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:00.771427    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gmwd" podUID="a7a85a73-907b-4a3b-bee4-b03ce0da2b6c"
Jul 07 15:01:00.771562 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:00.771547    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zs477" podUID="0ebbf4a6-429c-465d-916d-84567a210db5"
Jul 07 15:01:00.947945 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:00.947906    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xj52l" event={"ID":"ca0a1db1-0575-4ea6-a93a-6445a39114b2","Type":"ContainerStarted","Data":"f1831f52dbed4c90daf188f92c26f113bae7303ed454d3eb88a545b59e9eac9d"}
Jul 07 15:01:00.980378 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:00.980125    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xj52l" podStartSLOduration=7.519777718 podStartE2EDuration="27.980102259s" totalImagesPullingTime="20.460324541s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:00:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:00.980038973 +0000 UTC m=+28.776575727" watchObservedRunningTime="2026-07-07 15:01:00.980102259 +0000 UTC m=+28.776638983"
Jul 07 15:01:01.771291 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:01.771249    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:01:01.771477 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:01.771373    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zz5ql" podUID="31b4f468-dc43-4504-b8c1-3f6b91c08bc5"
Jul 07 15:01:01.950903 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:01.950867    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:01:01.951337 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:01.950983    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:01:01.951337 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:01.951020    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:01:01.968765 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:01.968730    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:01:01.969327 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:01.969312    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:01:02.667223 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:02.667187    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9gmwd"]
Jul 07 15:01:02.667401 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:02.667327    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:01:02.667466 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:02.667431    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gmwd" podUID="a7a85a73-907b-4a3b-bee4-b03ce0da2b6c"
Jul 07 15:01:02.671005 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:02.670778    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zz5ql"]
Jul 07 15:01:02.671005 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:02.670904    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:01:02.671005 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:02.670975    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zz5ql" podUID="31b4f468-dc43-4504-b8c1-3f6b91c08bc5"
Jul 07 15:01:02.672676 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:02.672652    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zs477"]
Jul 07 15:01:02.672799 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:02.672760    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:01:02.672918 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:02.672893    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zs477" podUID="0ebbf4a6-429c-465d-916d-84567a210db5"
Jul 07 15:01:02.954334 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:02.954249    2567 generic.go:356] "Generic (PLEG): container finished" podID="c6eef884-173e-4c5b-890d-3409f42fb0a3" containerID="2a02edc908fe34068349f44c73d0e081d28e0016c8f1a205b1c15c123b3062a9" exitCode=0
Jul 07 15:01:02.954823 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:02.954329    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5r27h" event={"ID":"c6eef884-173e-4c5b-890d-3409f42fb0a3","Type":"ContainerDied","Data":"2a02edc908fe34068349f44c73d0e081d28e0016c8f1a205b1c15c123b3062a9"}
Jul 07 15:01:03.771345 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:03.771305    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:01:03.771556 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:03.771419    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zz5ql" podUID="31b4f468-dc43-4504-b8c1-3f6b91c08bc5"
Jul 07 15:01:04.771726 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:04.771617    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:01:04.771726 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:04.771680    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:01:04.772134 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:04.771777    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zs477" podUID="0ebbf4a6-429c-465d-916d-84567a210db5"
Jul 07 15:01:04.772134 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:04.771928    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gmwd" podUID="a7a85a73-907b-4a3b-bee4-b03ce0da2b6c"
Jul 07 15:01:04.960666 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:04.960438    2567 generic.go:356] "Generic (PLEG): container finished" podID="c6eef884-173e-4c5b-890d-3409f42fb0a3" containerID="20cb290762c253a9015e70eb799d16b4a1a4d3c59487e312af60718879f1d18c" exitCode=0
Jul 07 15:01:04.960815 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:04.960522    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5r27h" event={"ID":"c6eef884-173e-4c5b-890d-3409f42fb0a3","Type":"ContainerDied","Data":"20cb290762c253a9015e70eb799d16b4a1a4d3c59487e312af60718879f1d18c"}
Jul 07 15:01:05.771535 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:05.771505    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:01:05.771710 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:05.771617    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zz5ql" podUID="31b4f468-dc43-4504-b8c1-3f6b91c08bc5"
Jul 07 15:01:06.404370 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:06.404342    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-original-pull-secret\") pod \"global-pull-secret-syncer-zz5ql\" (UID: \"31b4f468-dc43-4504-b8c1-3f6b91c08bc5\") " pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:01:06.404674 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:06.404441    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 15:01:06.404674 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:06.404489    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-original-pull-secret podName:31b4f468-dc43-4504-b8c1-3f6b91c08bc5 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:38.404478015 +0000 UTC m=+66.201014722 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-original-pull-secret") pod "global-pull-secret-syncer-zz5ql" (UID: "31b4f468-dc43-4504-b8c1-3f6b91c08bc5") : object "kube-system"/"original-pull-secret" not registered
Jul 07 15:01:06.504977 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:06.504944    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ebbf4a6-429c-465d-916d-84567a210db5-metrics-certs\") pod \"network-metrics-daemon-zs477\" (UID: \"0ebbf4a6-429c-465d-916d-84567a210db5\") " pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:01:06.505135 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:06.505068    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 15:01:06.505135 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:06.505126    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ebbf4a6-429c-465d-916d-84567a210db5-metrics-certs podName:0ebbf4a6-429c-465d-916d-84567a210db5 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:38.50511278 +0000 UTC m=+66.301649486 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ebbf4a6-429c-465d-916d-84567a210db5-metrics-certs") pod "network-metrics-daemon-zs477" (UID: "0ebbf4a6-429c-465d-916d-84567a210db5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 15:01:06.605782 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:06.605693    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvn2q\" (UniqueName: \"kubernetes.io/projected/a7a85a73-907b-4a3b-bee4-b03ce0da2b6c-kube-api-access-jvn2q\") pod \"network-check-target-9gmwd\" (UID: \"a7a85a73-907b-4a3b-bee4-b03ce0da2b6c\") " pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:01:06.605944 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:06.605818    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 15:01:06.605944 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:06.605855    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 15:01:06.605944 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:06.605865    2567 projected.go:196] Error preparing data for projected volume kube-api-access-jvn2q for pod openshift-network-diagnostics/network-check-target-9gmwd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 15:01:06.605944 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:06.605911    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7a85a73-907b-4a3b-bee4-b03ce0da2b6c-kube-api-access-jvn2q podName:a7a85a73-907b-4a3b-bee4-b03ce0da2b6c nodeName:}" failed. No retries permitted until 2026-07-07 15:01:38.605897967 +0000 UTC m=+66.402434685 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jvn2q" (UniqueName: "kubernetes.io/projected/a7a85a73-907b-4a3b-bee4-b03ce0da2b6c-kube-api-access-jvn2q") pod "network-check-target-9gmwd" (UID: "a7a85a73-907b-4a3b-bee4-b03ce0da2b6c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 15:01:06.771950 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:06.771922    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:01:06.772097 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:06.771932    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:01:06.772097 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:06.772031    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zs477" podUID="0ebbf4a6-429c-465d-916d-84567a210db5"
Jul 07 15:01:06.772188 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:06.772105    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9gmwd" podUID="a7a85a73-907b-4a3b-bee4-b03ce0da2b6c"
Jul 07 15:01:06.967645 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:06.967557    2567 generic.go:356] "Generic (PLEG): container finished" podID="c6eef884-173e-4c5b-890d-3409f42fb0a3" containerID="3265a83a7255074fbe14c6bffa34d152f56d528e8cebff929f656768cd347585" exitCode=0
Jul 07 15:01:06.967645 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:06.967613    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5r27h" event={"ID":"c6eef884-173e-4c5b-890d-3409f42fb0a3","Type":"ContainerDied","Data":"3265a83a7255074fbe14c6bffa34d152f56d528e8cebff929f656768cd347585"}
Jul 07 15:01:07.771458 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:07.771426    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:01:07.771902 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:07.771524    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zz5ql" podUID="31b4f468-dc43-4504-b8c1-3f6b91c08bc5"
Jul 07 15:01:07.991518 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:07.991484    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-233.ec2.internal" event="NodeReady"
Jul 07 15:01:07.991683 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:07.991619    2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 07 15:01:08.032743 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.032602    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qt79g"]
Jul 07 15:01:08.034491 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.034461    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-58948c8cff-pclzr"]
Jul 07 15:01:08.034621 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.034606    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qt79g"
Jul 07 15:01:08.036706 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.036686    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:08.036941 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.036920    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6cfbcb4769-87kk4"]
Jul 07 15:01:08.038783 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.038763    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-d5g6n"]
Jul 07 15:01:08.038913 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.038899    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:08.040461 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.040442    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tfz7j"]
Jul 07 15:01:08.040557 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.040543    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-d5g6n"
Jul 07 15:01:08.041329 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.041308    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 07 15:01:08.041586 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.041456    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xfh92\""
Jul 07 15:01:08.041932 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.041916    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 07 15:01:08.042382 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.042114    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 07 15:01:08.042382 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.042150    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 07 15:01:08.042519 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.042458    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f4wfx"]
Jul 07 15:01:08.042595 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.042577    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tfz7j"
Jul 07 15:01:08.043004 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.042980    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 07 15:01:08.043109 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.043029    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 07 15:01:08.045914 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.044650    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 07 15:01:08.045914 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.044674    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ds29c"]
Jul 07 15:01:08.045914 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.044969    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-ts8vc\""
Jul 07 15:01:08.045914 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.045165    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 07 15:01:08.047986 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.046680    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 07 15:01:08.047986 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.047510    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 07 15:01:08.048437 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.048419    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f4wfx"
Jul 07 15:01:08.048517 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.048497    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 07 15:01:08.049434 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.049411    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-cz626"]
Jul 07 15:01:08.049575 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.049558    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ds29c"
Jul 07 15:01:08.051504 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.051484    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-5r9rp"]
Jul 07 15:01:08.051690 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.051602    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-cz626"
Jul 07 15:01:08.053499 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.053482    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-5r9rp"
Jul 07 15:01:08.054882 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.054856    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 07 15:01:08.054882 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.054867    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 07 15:01:08.055336 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.055067    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 07 15:01:08.055336 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.055128    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 07 15:01:08.055686 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.055461    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 07 15:01:08.055772 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.055710    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-b9tdv\""
Jul 07 15:01:08.055855 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.055807    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 07 15:01:08.055914 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.055809    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 07 15:01:08.056091 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.056067    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 07 15:01:08.056442 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.056208    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 07 15:01:08.056442 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.056366    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-8z6h4\""
Jul 07 15:01:08.056585 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.056523    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-t8hm9\""
Jul 07 15:01:08.057492 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.057158    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qt79g"]
Jul 07 15:01:08.057932 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.057912    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 07 15:01:08.058655 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.058637    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 07 15:01:08.060862 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.060040    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 07 15:01:08.060862 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.060263    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 07 15:01:08.060862 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.060430    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tfz7j"]
Jul 07 15:01:08.060862 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.060475    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 07 15:01:08.060862 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.060518    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-2qtwl\""
Jul 07 15:01:08.060862 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.060679    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 07 15:01:08.060862 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.060758    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 07 15:01:08.061229 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.061065    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 07 15:01:08.061229 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.061091    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f4wfx"]
Jul 07 15:01:08.061654 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.061428    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 07 15:01:08.061654 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.061534    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 07 15:01:08.062377 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.061952    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 07 15:01:08.062377 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.062312    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-2bl6v\""
Jul 07 15:01:08.065393 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.064820    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-grc5d\""
Jul 07 15:01:08.065393 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.064899    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 07 15:01:08.065393 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.065234    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 07 15:01:08.066471 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.066451    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-gqnsw\""
Jul 07 15:01:08.067417 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.066692    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 07 15:01:08.074944 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.074925    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 07 15:01:08.078611 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.078363    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 07 15:01:08.080499 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.079901    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-d5g6n"]
Jul 07 15:01:08.081200 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.081182    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ds29c"]
Jul 07 15:01:08.081501 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.081483    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-58948c8cff-pclzr"]
Jul 07 15:01:08.082757 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.082735    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 07 15:01:08.083094 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.083077    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-cz626"]
Jul 07 15:01:08.084293 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.084276    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-rpzmw"]
Jul 07 15:01:08.087401 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.087378    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cfbcb4769-87kk4"]
Jul 07 15:01:08.087561 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.087476    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-rpzmw"
Jul 07 15:01:08.091597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.090697    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 15:01:08.097162 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.097142    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-gl24t\""
Jul 07 15:01:08.097447 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.097158    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 15:01:08.098903 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.098880    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-5r9rp"]
Jul 07 15:01:08.099472 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.099347    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-rpzmw"]
Jul 07 15:01:08.111514 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.111491    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-w9blj"]
Jul 07 15:01:08.113770 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.113716    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-w9blj"
Jul 07 15:01:08.114312 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.114289    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/56f88c5a-6833-4eab-b858-8c4b163fbdcb-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-qt79g\" (UID: \"56f88c5a-6833-4eab-b858-8c4b163fbdcb\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qt79g"
Jul 07 15:01:08.114392 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.114331    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-ca-trust-extracted\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:08.114392 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.114361    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9lb6\" (UniqueName: \"kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-kube-api-access-n9lb6\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:08.114473 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.114411    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/422c8ccc-828c-4491-a88b-76cd6973e056-tmp\") pod \"insights-operator-5c847dcfc4-d5g6n\" (UID: \"422c8ccc-828c-4491-a88b-76cd6973e056\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5g6n"
Jul 07 15:01:08.114473 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.114458    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-tls\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:08.114535 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.114525    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/44e4a5b8-14bf-482c-9297-f2809ddb36c1-default-certificate\") pod \"router-default-58948c8cff-pclzr\" (UID: \"44e4a5b8-14bf-482c-9297-f2809ddb36c1\") " pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:08.114569 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.114549    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44e4a5b8-14bf-482c-9297-f2809ddb36c1-service-ca-bundle\") pod \"router-default-58948c8cff-pclzr\" (UID: \"44e4a5b8-14bf-482c-9297-f2809ddb36c1\") " pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:08.114600 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.114576    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/56f88c5a-6833-4eab-b858-8c4b163fbdcb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-qt79g\" (UID: \"56f88c5a-6833-4eab-b858-8c4b163fbdcb\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qt79g"
Jul 07 15:01:08.114633 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.114602    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/44e4a5b8-14bf-482c-9297-f2809ddb36c1-stats-auth\") pod \"router-default-58948c8cff-pclzr\" (UID: \"44e4a5b8-14bf-482c-9297-f2809ddb36c1\") " pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:08.114633 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.114626    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckhlf\" (UniqueName: \"kubernetes.io/projected/44e4a5b8-14bf-482c-9297-f2809ddb36c1-kube-api-access-ckhlf\") pod \"router-default-58948c8cff-pclzr\" (UID: \"44e4a5b8-14bf-482c-9297-f2809ddb36c1\") " pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:08.114690 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.114653    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-certificates\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:08.114738 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.114686    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-trusted-ca\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:08.114738 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.114718    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-bound-sa-token\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:08.114858 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.114757    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/422c8ccc-828c-4491-a88b-76cd6973e056-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-d5g6n\" (UID: \"422c8ccc-828c-4491-a88b-76cd6973e056\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5g6n"
Jul 07 15:01:08.114858 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.114787    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxsfn\" (UniqueName: \"kubernetes.io/projected/56f88c5a-6833-4eab-b858-8c4b163fbdcb-kube-api-access-mxsfn\") pod \"cluster-monitoring-operator-64786d9cbb-qt79g\" (UID: \"56f88c5a-6833-4eab-b858-8c4b163fbdcb\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qt79g"
Jul 07 15:01:08.114858 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.114812    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44e4a5b8-14bf-482c-9297-f2809ddb36c1-metrics-certs\") pod \"router-default-58948c8cff-pclzr\" (UID: \"44e4a5b8-14bf-482c-9297-f2809ddb36c1\") " pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:08.115011 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.114867    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/422c8ccc-828c-4491-a88b-76cd6973e056-snapshots\") pod \"insights-operator-5c847dcfc4-d5g6n\" (UID: \"422c8ccc-828c-4491-a88b-76cd6973e056\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5g6n"
Jul 07 15:01:08.115011 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.114907    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/422c8ccc-828c-4491-a88b-76cd6973e056-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-d5g6n\" (UID: \"422c8ccc-828c-4491-a88b-76cd6973e056\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5g6n"
Jul 07 15:01:08.115011 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.114940    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-installation-pull-secrets\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:08.115011 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.114955    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5lv7\" (UniqueName: \"kubernetes.io/projected/422c8ccc-828c-4491-a88b-76cd6973e056-kube-api-access-l5lv7\") pod \"insights-operator-5c847dcfc4-d5g6n\" (UID: \"422c8ccc-828c-4491-a88b-76cd6973e056\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5g6n"
Jul 07 15:01:08.115011 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.114977    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-image-registry-private-configuration\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:08.115011 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.114998    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/422c8ccc-828c-4491-a88b-76cd6973e056-serving-cert\") pod \"insights-operator-5c847dcfc4-d5g6n\" (UID: \"422c8ccc-828c-4491-a88b-76cd6973e056\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5g6n"
Jul 07 15:01:08.121412 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.121191    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 07 15:01:08.121412 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.121199    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 07 15:01:08.121663 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.121644    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mt4rv\""
Jul 07 15:01:08.122031 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.122010    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 07 15:01:08.127038 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.127018    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-w9blj"]
Jul 07 15:01:08.156990 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.156957    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rlrhq"]
Jul 07 15:01:08.158983 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.158961    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rlrhq"
Jul 07 15:01:08.163783 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.163762    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 07 15:01:08.164162 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.164146    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6qs2h\""
Jul 07 15:01:08.164392 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.164371    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 07 15:01:08.171328 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.171306    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rlrhq"]
Jul 07 15:01:08.216193 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.216158    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-tls\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:08.216367 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.216203    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlfgv\" (UniqueName: \"kubernetes.io/projected/34299ca2-ae50-4512-a950-117fa8d0c013-kube-api-access-nlfgv\") pod \"network-check-source-989f8b8dc-rpzmw\" (UID: \"34299ca2-ae50-4512-a950-117fa8d0c013\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-rpzmw"
Jul 07 15:01:08.216367 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.216232    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckhlf\" (UniqueName: \"kubernetes.io/projected/44e4a5b8-14bf-482c-9297-f2809ddb36c1-kube-api-access-ckhlf\") pod \"router-default-58948c8cff-pclzr\" (UID: \"44e4a5b8-14bf-482c-9297-f2809ddb36c1\") " pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:08.216367 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:08.216335    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 15:01:08.216367 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:08.216356    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cfbcb4769-87kk4: secret "image-registry-tls" not found
Jul 07 15:01:08.216576 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.216379    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-certificates\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:08.216576 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:08.216407    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-tls podName:d050a3c0-f99d-4e6d-99f3-03ce677e7fa0 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:08.716390052 +0000 UTC m=+36.512926761 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-tls") pod "image-registry-6cfbcb4769-87kk4" (UID: "d050a3c0-f99d-4e6d-99f3-03ce677e7fa0") : secret "image-registry-tls" not found
Jul 07 15:01:08.216576 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.216441    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/44e4a5b8-14bf-482c-9297-f2809ddb36c1-default-certificate\") pod \"router-default-58948c8cff-pclzr\" (UID: \"44e4a5b8-14bf-482c-9297-f2809ddb36c1\") " pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:08.216576 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.216469    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44e4a5b8-14bf-482c-9297-f2809ddb36c1-service-ca-bundle\") pod \"router-default-58948c8cff-pclzr\" (UID: \"44e4a5b8-14bf-482c-9297-f2809ddb36c1\") " pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:08.216576 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.216490    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-279p5\" (UniqueName: \"kubernetes.io/projected/dbe1e84c-636f-4d7e-adcd-67aea6cd7a34-kube-api-access-279p5\") pod \"ingress-canary-w9blj\" (UID: \"dbe1e84c-636f-4d7e-adcd-67aea6cd7a34\") " pod="openshift-ingress-canary/ingress-canary-w9blj"
Jul 07 15:01:08.216576 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.216514    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/422c8ccc-828c-4491-a88b-76cd6973e056-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-d5g6n\" (UID: \"422c8ccc-828c-4491-a88b-76cd6973e056\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5g6n"
Jul 07 15:01:08.216576 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.216541    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1c51cd93-b0ea-4aa1-8678-a4c833b2d37e-serving-cert\") pod \"console-operator-68b757fb67-5r9rp\" (UID: \"1c51cd93-b0ea-4aa1-8678-a4c833b2d37e\") " pod="openshift-console-operator/console-operator-68b757fb67-5r9rp"
Jul 07 15:01:08.216576 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.216564    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/73d589b0-98d3-40b3-8179-27d4ff10efbf-config\") pod \"service-ca-operator-5d46d9ffd4-cz626\" (UID: \"73d589b0-98d3-40b3-8179-27d4ff10efbf\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-cz626"
Jul 07 15:01:08.216999 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.216594    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/44e4a5b8-14bf-482c-9297-f2809ddb36c1-stats-auth\") pod \"router-default-58948c8cff-pclzr\" (UID: \"44e4a5b8-14bf-482c-9297-f2809ddb36c1\") " pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:08.216999 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:08.216625    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/44e4a5b8-14bf-482c-9297-f2809ddb36c1-service-ca-bundle podName:44e4a5b8-14bf-482c-9297-f2809ddb36c1 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:08.716610959 +0000 UTC m=+36.513147684 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/44e4a5b8-14bf-482c-9297-f2809ddb36c1-service-ca-bundle") pod "router-default-58948c8cff-pclzr" (UID: "44e4a5b8-14bf-482c-9297-f2809ddb36c1") : configmap references non-existent config key: service-ca.crt
Jul 07 15:01:08.216999 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.216647    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/422c8ccc-828c-4491-a88b-76cd6973e056-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-d5g6n\" (UID: \"422c8ccc-828c-4491-a88b-76cd6973e056\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5g6n"
Jul 07 15:01:08.216999 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.216677    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/719d2a5c-4213-4e2a-bce0-9b29e0ad02c0-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-tfz7j\" (UID: \"719d2a5c-4213-4e2a-bce0-9b29e0ad02c0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tfz7j"
Jul 07 15:01:08.216999 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.216706    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44e4a5b8-14bf-482c-9297-f2809ddb36c1-metrics-certs\") pod \"router-default-58948c8cff-pclzr\" (UID: \"44e4a5b8-14bf-482c-9297-f2809ddb36c1\") " pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:08.216999 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.216729    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/422c8ccc-828c-4491-a88b-76cd6973e056-snapshots\") pod \"insights-operator-5c847dcfc4-d5g6n\" (UID: \"422c8ccc-828c-4491-a88b-76cd6973e056\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5g6n"
Jul 07 15:01:08.216999 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.216769    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blb9k\" (UniqueName: \"kubernetes.io/projected/9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118-kube-api-access-blb9k\") pod \"cluster-samples-operator-56fcf86bf-f4wfx\" (UID: \"9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f4wfx"
Jul 07 15:01:08.216999 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.216801    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-installation-pull-secrets\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:08.216999 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.216826    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5lv7\" (UniqueName: \"kubernetes.io/projected/422c8ccc-828c-4491-a88b-76cd6973e056-kube-api-access-l5lv7\") pod \"insights-operator-5c847dcfc4-d5g6n\" (UID: \"422c8ccc-828c-4491-a88b-76cd6973e056\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5g6n"
Jul 07 15:01:08.216999 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.216873    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-image-registry-private-configuration\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:08.216999 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.216893    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dbe1e84c-636f-4d7e-adcd-67aea6cd7a34-cert\") pod \"ingress-canary-w9blj\" (UID: \"dbe1e84c-636f-4d7e-adcd-67aea6cd7a34\") " pod="openshift-ingress-canary/ingress-canary-w9blj"
Jul 07 15:01:08.216999 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.216913    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-ca-trust-extracted\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:08.216999 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.216979    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr28f\" (UniqueName: \"kubernetes.io/projected/719d2a5c-4213-4e2a-bce0-9b29e0ad02c0-kube-api-access-fr28f\") pod \"kube-storage-version-migrator-operator-846bf668f7-tfz7j\" (UID: \"719d2a5c-4213-4e2a-bce0-9b29e0ad02c0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tfz7j"
Jul 07 15:01:08.217603 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.217008    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/73d589b0-98d3-40b3-8179-27d4ff10efbf-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-cz626\" (UID: \"73d589b0-98d3-40b3-8179-27d4ff10efbf\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-cz626"
Jul 07 15:01:08.217603 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.217035    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-bound-sa-token\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:08.217603 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.217037    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-certificates\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:08.217603 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.217089    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1c51cd93-b0ea-4aa1-8678-a4c833b2d37e-trusted-ca\") pod \"console-operator-68b757fb67-5r9rp\" (UID: \"1c51cd93-b0ea-4aa1-8678-a4c833b2d37e\") " pod="openshift-console-operator/console-operator-68b757fb67-5r9rp"
Jul 07 15:01:08.217603 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.217126    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-trusted-ca\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:08.217603 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.217154    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5sxp\" (UniqueName: \"kubernetes.io/projected/f3ad7a15-08da-4441-8d40-2931adbefc20-kube-api-access-n5sxp\") pod \"volume-data-source-validator-5d456fdbcd-ds29c\" (UID: \"f3ad7a15-08da-4441-8d40-2931adbefc20\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ds29c"
Jul 07 15:01:08.217603 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.217183    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/56f88c5a-6833-4eab-b858-8c4b163fbdcb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-qt79g\" (UID: \"56f88c5a-6833-4eab-b858-8c4b163fbdcb\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qt79g"
Jul 07 15:01:08.217603 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.217218    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxsfn\" (UniqueName: \"kubernetes.io/projected/56f88c5a-6833-4eab-b858-8c4b163fbdcb-kube-api-access-mxsfn\") pod \"cluster-monitoring-operator-64786d9cbb-qt79g\" (UID: \"56f88c5a-6833-4eab-b858-8c4b163fbdcb\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qt79g"
Jul 07 15:01:08.217603 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.217242    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/719d2a5c-4213-4e2a-bce0-9b29e0ad02c0-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-tfz7j\" (UID: \"719d2a5c-4213-4e2a-bce0-9b29e0ad02c0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tfz7j"
Jul 07 15:01:08.217603 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.217320    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9zcn\" (UniqueName: \"kubernetes.io/projected/73d589b0-98d3-40b3-8179-27d4ff10efbf-kube-api-access-n9zcn\") pod \"service-ca-operator-5d46d9ffd4-cz626\" (UID: \"73d589b0-98d3-40b3-8179-27d4ff10efbf\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-cz626"
Jul 07 15:01:08.217603 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:08.217556    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 15:01:08.217603 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.217566    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/422c8ccc-828c-4491-a88b-76cd6973e056-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-d5g6n\" (UID: \"422c8ccc-828c-4491-a88b-76cd6973e056\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5g6n"
Jul 07 15:01:08.217603 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:08.217605    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44e4a5b8-14bf-482c-9297-f2809ddb36c1-metrics-certs podName:44e4a5b8-14bf-482c-9297-f2809ddb36c1 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:08.717590345 +0000 UTC m=+36.514127065 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/44e4a5b8-14bf-482c-9297-f2809ddb36c1-metrics-certs") pod "router-default-58948c8cff-pclzr" (UID: "44e4a5b8-14bf-482c-9297-f2809ddb36c1") : secret "router-metrics-certs-default" not found
Jul 07 15:01:08.218283 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.217638    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/56f88c5a-6833-4eab-b858-8c4b163fbdcb-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-qt79g\" (UID: \"56f88c5a-6833-4eab-b858-8c4b163fbdcb\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qt79g"
Jul 07 15:01:08.218283 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.217657    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9lb6\" (UniqueName: \"kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-kube-api-access-n9lb6\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:08.218283 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:08.218000    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 15:01:08.218283 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:08.218040    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56f88c5a-6833-4eab-b858-8c4b163fbdcb-cluster-monitoring-operator-tls podName:56f88c5a-6833-4eab-b858-8c4b163fbdcb nodeName:}" failed. No retries permitted until 2026-07-07 15:01:08.718027784 +0000 UTC m=+36.514564505 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/56f88c5a-6833-4eab-b858-8c4b163fbdcb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-qt79g" (UID: "56f88c5a-6833-4eab-b858-8c4b163fbdcb") : secret "cluster-monitoring-operator-tls" not found
Jul 07 15:01:08.218283 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.218068    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/422c8ccc-828c-4491-a88b-76cd6973e056-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-d5g6n\" (UID: \"422c8ccc-828c-4491-a88b-76cd6973e056\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5g6n"
Jul 07 15:01:08.218528 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.218316    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-ca-trust-extracted\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:08.218528 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.218372    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/56f88c5a-6833-4eab-b858-8c4b163fbdcb-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-qt79g\" (UID: \"56f88c5a-6833-4eab-b858-8c4b163fbdcb\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qt79g"
Jul 07 15:01:08.218528 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.218406    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1c51cd93-b0ea-4aa1-8678-a4c833b2d37e-config\") pod \"console-operator-68b757fb67-5r9rp\" (UID: \"1c51cd93-b0ea-4aa1-8678-a4c833b2d37e\") " pod="openshift-console-operator/console-operator-68b757fb67-5r9rp"
Jul 07 15:01:08.218528 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.218439    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbvql\" (UniqueName: \"kubernetes.io/projected/1c51cd93-b0ea-4aa1-8678-a4c833b2d37e-kube-api-access-fbvql\") pod \"console-operator-68b757fb67-5r9rp\" (UID: \"1c51cd93-b0ea-4aa1-8678-a4c833b2d37e\") " pod="openshift-console-operator/console-operator-68b757fb67-5r9rp"
Jul 07 15:01:08.218528 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.218457    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-f4wfx\" (UID: \"9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f4wfx"
Jul 07 15:01:08.218528 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.218495    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/422c8ccc-828c-4491-a88b-76cd6973e056-tmp\") pod \"insights-operator-5c847dcfc4-d5g6n\" (UID: \"422c8ccc-828c-4491-a88b-76cd6973e056\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5g6n"
Jul 07 15:01:08.218528 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.218528    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/422c8ccc-828c-4491-a88b-76cd6973e056-serving-cert\") pod \"insights-operator-5c847dcfc4-d5g6n\" (UID: \"422c8ccc-828c-4491-a88b-76cd6973e056\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5g6n"
Jul 07 15:01:08.218887 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.218668    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-trusted-ca\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:08.218887 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.218705    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/422c8ccc-828c-4491-a88b-76cd6973e056-snapshots\") pod \"insights-operator-5c847dcfc4-d5g6n\" (UID: \"422c8ccc-828c-4491-a88b-76cd6973e056\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5g6n"
Jul 07 15:01:08.219005 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.218945    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/422c8ccc-828c-4491-a88b-76cd6973e056-tmp\") pod \"insights-operator-5c847dcfc4-d5g6n\" (UID: \"422c8ccc-828c-4491-a88b-76cd6973e056\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5g6n"
Jul 07 15:01:08.221991 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.221968    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/422c8ccc-828c-4491-a88b-76cd6973e056-serving-cert\") pod \"insights-operator-5c847dcfc4-d5g6n\" (UID: \"422c8ccc-828c-4491-a88b-76cd6973e056\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5g6n"
Jul 07 15:01:08.222111 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.222021    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/44e4a5b8-14bf-482c-9297-f2809ddb36c1-stats-auth\") pod \"router-default-58948c8cff-pclzr\" (UID: \"44e4a5b8-14bf-482c-9297-f2809ddb36c1\") " pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:08.222111 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.222028    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-image-registry-private-configuration\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:08.222111 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.222082    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/44e4a5b8-14bf-482c-9297-f2809ddb36c1-default-certificate\") pod \"router-default-58948c8cff-pclzr\" (UID: \"44e4a5b8-14bf-482c-9297-f2809ddb36c1\") " pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:08.222261 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.222230    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-installation-pull-secrets\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:08.232848 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.232785    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckhlf\" (UniqueName: \"kubernetes.io/projected/44e4a5b8-14bf-482c-9297-f2809ddb36c1-kube-api-access-ckhlf\") pod \"router-default-58948c8cff-pclzr\" (UID: \"44e4a5b8-14bf-482c-9297-f2809ddb36c1\") " pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:08.235116 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.235073    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9lb6\" (UniqueName: \"kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-kube-api-access-n9lb6\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:08.235226 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.235123    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5lv7\" (UniqueName: \"kubernetes.io/projected/422c8ccc-828c-4491-a88b-76cd6973e056-kube-api-access-l5lv7\") pod \"insights-operator-5c847dcfc4-d5g6n\" (UID: \"422c8ccc-828c-4491-a88b-76cd6973e056\") " pod="openshift-insights/insights-operator-5c847dcfc4-d5g6n"
Jul 07 15:01:08.235679 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.235655    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-bound-sa-token\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:08.237202 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.237179    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxsfn\" (UniqueName: \"kubernetes.io/projected/56f88c5a-6833-4eab-b858-8c4b163fbdcb-kube-api-access-mxsfn\") pod \"cluster-monitoring-operator-64786d9cbb-qt79g\" (UID: \"56f88c5a-6833-4eab-b858-8c4b163fbdcb\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qt79g"
Jul 07 15:01:08.319621 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.319585    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/73d589b0-98d3-40b3-8179-27d4ff10efbf-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-cz626\" (UID: \"73d589b0-98d3-40b3-8179-27d4ff10efbf\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-cz626"
Jul 07 15:01:08.319793 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.319663    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1c51cd93-b0ea-4aa1-8678-a4c833b2d37e-trusted-ca\") pod \"console-operator-68b757fb67-5r9rp\" (UID: \"1c51cd93-b0ea-4aa1-8678-a4c833b2d37e\") " pod="openshift-console-operator/console-operator-68b757fb67-5r9rp"
Jul 07 15:01:08.319793 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.319713    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5sxp\" (UniqueName: \"kubernetes.io/projected/f3ad7a15-08da-4441-8d40-2931adbefc20-kube-api-access-n5sxp\") pod \"volume-data-source-validator-5d456fdbcd-ds29c\" (UID: \"f3ad7a15-08da-4441-8d40-2931adbefc20\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ds29c"
Jul 07 15:01:08.319793 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.319750    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdrgn\" (UniqueName: \"kubernetes.io/projected/23819f39-1440-4b6c-90fa-6db19eb48b74-kube-api-access-hdrgn\") pod \"dns-default-rlrhq\" (UID: \"23819f39-1440-4b6c-90fa-6db19eb48b74\") " pod="openshift-dns/dns-default-rlrhq"
Jul 07 15:01:08.319991 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.319799    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/719d2a5c-4213-4e2a-bce0-9b29e0ad02c0-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-tfz7j\" (UID: \"719d2a5c-4213-4e2a-bce0-9b29e0ad02c0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tfz7j"
Jul 07 15:01:08.319991 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.319854    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9zcn\" (UniqueName: \"kubernetes.io/projected/73d589b0-98d3-40b3-8179-27d4ff10efbf-kube-api-access-n9zcn\") pod \"service-ca-operator-5d46d9ffd4-cz626\" (UID: \"73d589b0-98d3-40b3-8179-27d4ff10efbf\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-cz626"
Jul 07 15:01:08.319991 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.319897    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23819f39-1440-4b6c-90fa-6db19eb48b74-tmp-dir\") pod \"dns-default-rlrhq\" (UID: \"23819f39-1440-4b6c-90fa-6db19eb48b74\") " pod="openshift-dns/dns-default-rlrhq"
Jul 07 15:01:08.320391 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.320360    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1c51cd93-b0ea-4aa1-8678-a4c833b2d37e-config\") pod \"console-operator-68b757fb67-5r9rp\" (UID: \"1c51cd93-b0ea-4aa1-8678-a4c833b2d37e\") " pod="openshift-console-operator/console-operator-68b757fb67-5r9rp"
Jul 07 15:01:08.320519 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.320430    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbvql\" (UniqueName: \"kubernetes.io/projected/1c51cd93-b0ea-4aa1-8678-a4c833b2d37e-kube-api-access-fbvql\") pod \"console-operator-68b757fb67-5r9rp\" (UID: \"1c51cd93-b0ea-4aa1-8678-a4c833b2d37e\") " pod="openshift-console-operator/console-operator-68b757fb67-5r9rp"
Jul 07 15:01:08.320519 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.320475    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-f4wfx\" (UID: \"9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f4wfx"
Jul 07 15:01:08.320620 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.320535    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlfgv\" (UniqueName: \"kubernetes.io/projected/34299ca2-ae50-4512-a950-117fa8d0c013-kube-api-access-nlfgv\") pod \"network-check-source-989f8b8dc-rpzmw\" (UID: \"34299ca2-ae50-4512-a950-117fa8d0c013\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-rpzmw"
Jul 07 15:01:08.320620 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.320601    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-279p5\" (UniqueName: \"kubernetes.io/projected/dbe1e84c-636f-4d7e-adcd-67aea6cd7a34-kube-api-access-279p5\") pod \"ingress-canary-w9blj\" (UID: \"dbe1e84c-636f-4d7e-adcd-67aea6cd7a34\") " pod="openshift-ingress-canary/ingress-canary-w9blj"
Jul 07 15:01:08.320715 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.320633    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1c51cd93-b0ea-4aa1-8678-a4c833b2d37e-serving-cert\") pod \"console-operator-68b757fb67-5r9rp\" (UID: \"1c51cd93-b0ea-4aa1-8678-a4c833b2d37e\") " pod="openshift-console-operator/console-operator-68b757fb67-5r9rp"
Jul 07 15:01:08.320715 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.320651    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/73d589b0-98d3-40b3-8179-27d4ff10efbf-config\") pod \"service-ca-operator-5d46d9ffd4-cz626\" (UID: \"73d589b0-98d3-40b3-8179-27d4ff10efbf\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-cz626"
Jul 07 15:01:08.320715 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.320676    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/719d2a5c-4213-4e2a-bce0-9b29e0ad02c0-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-tfz7j\" (UID: \"719d2a5c-4213-4e2a-bce0-9b29e0ad02c0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tfz7j"
Jul 07 15:01:08.320715 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.320707    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blb9k\" (UniqueName: \"kubernetes.io/projected/9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118-kube-api-access-blb9k\") pod \"cluster-samples-operator-56fcf86bf-f4wfx\" (UID: \"9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f4wfx"
Jul 07 15:01:08.320914 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.320728    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dbe1e84c-636f-4d7e-adcd-67aea6cd7a34-cert\") pod \"ingress-canary-w9blj\" (UID: \"dbe1e84c-636f-4d7e-adcd-67aea6cd7a34\") " pod="openshift-ingress-canary/ingress-canary-w9blj"
Jul 07 15:01:08.320914 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.320749    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23819f39-1440-4b6c-90fa-6db19eb48b74-metrics-tls\") pod \"dns-default-rlrhq\" (UID: \"23819f39-1440-4b6c-90fa-6db19eb48b74\") " pod="openshift-dns/dns-default-rlrhq"
Jul 07 15:01:08.320914 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.320808    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/23819f39-1440-4b6c-90fa-6db19eb48b74-config-volume\") pod \"dns-default-rlrhq\" (UID: \"23819f39-1440-4b6c-90fa-6db19eb48b74\") " pod="openshift-dns/dns-default-rlrhq"
Jul 07 15:01:08.320914 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.320898    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr28f\" (UniqueName: \"kubernetes.io/projected/719d2a5c-4213-4e2a-bce0-9b29e0ad02c0-kube-api-access-fr28f\") pod \"kube-storage-version-migrator-operator-846bf668f7-tfz7j\" (UID: \"719d2a5c-4213-4e2a-bce0-9b29e0ad02c0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tfz7j"
Jul 07 15:01:08.322435 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:08.321632    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 15:01:08.322435 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.321658    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1c51cd93-b0ea-4aa1-8678-a4c833b2d37e-trusted-ca\") pod \"console-operator-68b757fb67-5r9rp\" (UID: \"1c51cd93-b0ea-4aa1-8678-a4c833b2d37e\") " pod="openshift-console-operator/console-operator-68b757fb67-5r9rp"
Jul 07 15:01:08.322435 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:08.321704    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118-samples-operator-tls podName:9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:08.821679575 +0000 UTC m=+36.618216281 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-f4wfx" (UID: "9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118") : secret "samples-operator-tls" not found
Jul 07 15:01:08.322435 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.321886    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1c51cd93-b0ea-4aa1-8678-a4c833b2d37e-config\") pod \"console-operator-68b757fb67-5r9rp\" (UID: \"1c51cd93-b0ea-4aa1-8678-a4c833b2d37e\") " pod="openshift-console-operator/console-operator-68b757fb67-5r9rp"
Jul 07 15:01:08.323728 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.323271    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/73d589b0-98d3-40b3-8179-27d4ff10efbf-config\") pod \"service-ca-operator-5d46d9ffd4-cz626\" (UID: \"73d589b0-98d3-40b3-8179-27d4ff10efbf\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-cz626"
Jul 07 15:01:08.323864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.323818    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/719d2a5c-4213-4e2a-bce0-9b29e0ad02c0-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-tfz7j\" (UID: \"719d2a5c-4213-4e2a-bce0-9b29e0ad02c0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tfz7j"
Jul 07 15:01:08.326733 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.326707    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/73d589b0-98d3-40b3-8179-27d4ff10efbf-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-cz626\" (UID: \"73d589b0-98d3-40b3-8179-27d4ff10efbf\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-cz626"
Jul 07 15:01:08.328209 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.328147    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dbe1e84c-636f-4d7e-adcd-67aea6cd7a34-cert\") pod \"ingress-canary-w9blj\" (UID: \"dbe1e84c-636f-4d7e-adcd-67aea6cd7a34\") " pod="openshift-ingress-canary/ingress-canary-w9blj"
Jul 07 15:01:08.328329 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.328302    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1c51cd93-b0ea-4aa1-8678-a4c833b2d37e-serving-cert\") pod \"console-operator-68b757fb67-5r9rp\" (UID: \"1c51cd93-b0ea-4aa1-8678-a4c833b2d37e\") " pod="openshift-console-operator/console-operator-68b757fb67-5r9rp"
Jul 07 15:01:08.329069 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.329048    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/719d2a5c-4213-4e2a-bce0-9b29e0ad02c0-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-tfz7j\" (UID: \"719d2a5c-4213-4e2a-bce0-9b29e0ad02c0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tfz7j"
Jul 07 15:01:08.330529 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.330509    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr28f\" (UniqueName: \"kubernetes.io/projected/719d2a5c-4213-4e2a-bce0-9b29e0ad02c0-kube-api-access-fr28f\") pod \"kube-storage-version-migrator-operator-846bf668f7-tfz7j\" (UID: \"719d2a5c-4213-4e2a-bce0-9b29e0ad02c0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tfz7j"
Jul 07 15:01:08.338107 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.338080    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbvql\" (UniqueName: \"kubernetes.io/projected/1c51cd93-b0ea-4aa1-8678-a4c833b2d37e-kube-api-access-fbvql\") pod \"console-operator-68b757fb67-5r9rp\" (UID: \"1c51cd93-b0ea-4aa1-8678-a4c833b2d37e\") " pod="openshift-console-operator/console-operator-68b757fb67-5r9rp"
Jul 07 15:01:08.338444 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.338423    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlfgv\" (UniqueName: \"kubernetes.io/projected/34299ca2-ae50-4512-a950-117fa8d0c013-kube-api-access-nlfgv\") pod \"network-check-source-989f8b8dc-rpzmw\" (UID: \"34299ca2-ae50-4512-a950-117fa8d0c013\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-rpzmw"
Jul 07 15:01:08.338875 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.338810    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9zcn\" (UniqueName: \"kubernetes.io/projected/73d589b0-98d3-40b3-8179-27d4ff10efbf-kube-api-access-n9zcn\") pod \"service-ca-operator-5d46d9ffd4-cz626\" (UID: \"73d589b0-98d3-40b3-8179-27d4ff10efbf\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-cz626"
Jul 07 15:01:08.338998 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.338981    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-279p5\" (UniqueName: \"kubernetes.io/projected/dbe1e84c-636f-4d7e-adcd-67aea6cd7a34-kube-api-access-279p5\") pod \"ingress-canary-w9blj\" (UID: \"dbe1e84c-636f-4d7e-adcd-67aea6cd7a34\") " pod="openshift-ingress-canary/ingress-canary-w9blj"
Jul 07 15:01:08.339063 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.339043    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5sxp\" (UniqueName: \"kubernetes.io/projected/f3ad7a15-08da-4441-8d40-2931adbefc20-kube-api-access-n5sxp\") pod \"volume-data-source-validator-5d456fdbcd-ds29c\" (UID: \"f3ad7a15-08da-4441-8d40-2931adbefc20\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ds29c"
Jul 07 15:01:08.340810 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.340785    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-blb9k\" (UniqueName: \"kubernetes.io/projected/9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118-kube-api-access-blb9k\") pod \"cluster-samples-operator-56fcf86bf-f4wfx\" (UID: \"9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f4wfx"
Jul 07 15:01:08.377224 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.377195    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-d5g6n"
Jul 07 15:01:08.387488 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.387461    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tfz7j"
Jul 07 15:01:08.403616 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.403448    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ds29c"
Jul 07 15:01:08.410589 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.410560    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-cz626"
Jul 07 15:01:08.418260 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.418046    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-5r9rp"
Jul 07 15:01:08.422526 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.421499    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23819f39-1440-4b6c-90fa-6db19eb48b74-metrics-tls\") pod \"dns-default-rlrhq\" (UID: \"23819f39-1440-4b6c-90fa-6db19eb48b74\") " pod="openshift-dns/dns-default-rlrhq"
Jul 07 15:01:08.422526 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.421554    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/23819f39-1440-4b6c-90fa-6db19eb48b74-config-volume\") pod \"dns-default-rlrhq\" (UID: \"23819f39-1440-4b6c-90fa-6db19eb48b74\") " pod="openshift-dns/dns-default-rlrhq"
Jul 07 15:01:08.422526 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.421626    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdrgn\" (UniqueName: \"kubernetes.io/projected/23819f39-1440-4b6c-90fa-6db19eb48b74-kube-api-access-hdrgn\") pod \"dns-default-rlrhq\" (UID: \"23819f39-1440-4b6c-90fa-6db19eb48b74\") " pod="openshift-dns/dns-default-rlrhq"
Jul 07 15:01:08.422526 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:08.421657    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 15:01:08.422526 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.421681    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23819f39-1440-4b6c-90fa-6db19eb48b74-tmp-dir\") pod \"dns-default-rlrhq\" (UID: \"23819f39-1440-4b6c-90fa-6db19eb48b74\") " pod="openshift-dns/dns-default-rlrhq"
Jul 07 15:01:08.422526 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:08.421725    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23819f39-1440-4b6c-90fa-6db19eb48b74-metrics-tls podName:23819f39-1440-4b6c-90fa-6db19eb48b74 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:08.92170367 +0000 UTC m=+36.718240381 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/23819f39-1440-4b6c-90fa-6db19eb48b74-metrics-tls") pod "dns-default-rlrhq" (UID: "23819f39-1440-4b6c-90fa-6db19eb48b74") : secret "dns-default-metrics-tls" not found
Jul 07 15:01:08.422526 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.422027    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23819f39-1440-4b6c-90fa-6db19eb48b74-tmp-dir\") pod \"dns-default-rlrhq\" (UID: \"23819f39-1440-4b6c-90fa-6db19eb48b74\") " pod="openshift-dns/dns-default-rlrhq"
Jul 07 15:01:08.422526 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.422479    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/23819f39-1440-4b6c-90fa-6db19eb48b74-config-volume\") pod \"dns-default-rlrhq\" (UID: \"23819f39-1440-4b6c-90fa-6db19eb48b74\") " pod="openshift-dns/dns-default-rlrhq"
Jul 07 15:01:08.426137 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.426113    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-rpzmw"
Jul 07 15:01:08.433140 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.432775    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-w9blj"
Jul 07 15:01:08.438086 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.438045    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdrgn\" (UniqueName: \"kubernetes.io/projected/23819f39-1440-4b6c-90fa-6db19eb48b74-kube-api-access-hdrgn\") pod \"dns-default-rlrhq\" (UID: \"23819f39-1440-4b6c-90fa-6db19eb48b74\") " pod="openshift-dns/dns-default-rlrhq"
Jul 07 15:01:08.660501 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.660280    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tfz7j"]
Jul 07 15:01:08.670716 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.670679    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-cz626"]
Jul 07 15:01:08.675474 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.674559    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ds29c"]
Jul 07 15:01:08.675542 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.675530    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-d5g6n"]
Jul 07 15:01:08.678046 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:01:08.678009    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73d589b0_98d3_40b3_8179_27d4ff10efbf.slice/crio-5fc01550d62874c7d67205b884b706d8db7fc91b391280a7b25127186caa71c1 WatchSource:0}: Error finding container 5fc01550d62874c7d67205b884b706d8db7fc91b391280a7b25127186caa71c1: Status 404 returned error can't find the container with id 5fc01550d62874c7d67205b884b706d8db7fc91b391280a7b25127186caa71c1
Jul 07 15:01:08.683705 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:01:08.683678    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod422c8ccc_828c_4491_a88b_76cd6973e056.slice/crio-8c10f09b178c3b2d2e35547b3466cb054fa4a2d444e0a56472bdd8998a3d96c4 WatchSource:0}: Error finding container 8c10f09b178c3b2d2e35547b3466cb054fa4a2d444e0a56472bdd8998a3d96c4: Status 404 returned error can't find the container with id 8c10f09b178c3b2d2e35547b3466cb054fa4a2d444e0a56472bdd8998a3d96c4
Jul 07 15:01:08.704214 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.704180    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-5r9rp"]
Jul 07 15:01:08.708713 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:01:08.708681    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c51cd93_b0ea_4aa1_8678_a4c833b2d37e.slice/crio-5e3bf3648a940e8724d40eb7123d45cafade66ab9f70d4b77347685fb34d979e WatchSource:0}: Error finding container 5e3bf3648a940e8724d40eb7123d45cafade66ab9f70d4b77347685fb34d979e: Status 404 returned error can't find the container with id 5e3bf3648a940e8724d40eb7123d45cafade66ab9f70d4b77347685fb34d979e
Jul 07 15:01:08.710916 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.710887    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-rpzmw"]
Jul 07 15:01:08.713653 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:01:08.713627    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34299ca2_ae50_4512_a950_117fa8d0c013.slice/crio-438eb662d359957395bed9b1abfc69457182fb1df535332bdf84cb9ccc786bac WatchSource:0}: Error finding container 438eb662d359957395bed9b1abfc69457182fb1df535332bdf84cb9ccc786bac: Status 404 returned error can't find the container with id 438eb662d359957395bed9b1abfc69457182fb1df535332bdf84cb9ccc786bac
Jul 07 15:01:08.720792 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.720771    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-w9blj"]
Jul 07 15:01:08.723427 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:01:08.723400    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbe1e84c_636f_4d7e_adcd_67aea6cd7a34.slice/crio-d77566911884e170dc45da831006a0690ce709d0f9fc0d39d50f73e01a9eb509 WatchSource:0}: Error finding container d77566911884e170dc45da831006a0690ce709d0f9fc0d39d50f73e01a9eb509: Status 404 returned error can't find the container with id d77566911884e170dc45da831006a0690ce709d0f9fc0d39d50f73e01a9eb509
Jul 07 15:01:08.727108 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.727085    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44e4a5b8-14bf-482c-9297-f2809ddb36c1-metrics-certs\") pod \"router-default-58948c8cff-pclzr\" (UID: \"44e4a5b8-14bf-482c-9297-f2809ddb36c1\") " pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:08.727215 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.727197    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/56f88c5a-6833-4eab-b858-8c4b163fbdcb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-qt79g\" (UID: \"56f88c5a-6833-4eab-b858-8c4b163fbdcb\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qt79g"
Jul 07 15:01:08.727275 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:08.727258    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 15:01:08.727327 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.727271    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-tls\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:08.727327 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:08.727322    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44e4a5b8-14bf-482c-9297-f2809ddb36c1-metrics-certs podName:44e4a5b8-14bf-482c-9297-f2809ddb36c1 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:09.727303007 +0000 UTC m=+37.523839719 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/44e4a5b8-14bf-482c-9297-f2809ddb36c1-metrics-certs") pod "router-default-58948c8cff-pclzr" (UID: "44e4a5b8-14bf-482c-9297-f2809ddb36c1") : secret "router-metrics-certs-default" not found
Jul 07 15:01:08.727437 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.727347    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44e4a5b8-14bf-482c-9297-f2809ddb36c1-service-ca-bundle\") pod \"router-default-58948c8cff-pclzr\" (UID: \"44e4a5b8-14bf-482c-9297-f2809ddb36c1\") " pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:08.727437 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:08.727367    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 15:01:08.727437 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:08.727424    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56f88c5a-6833-4eab-b858-8c4b163fbdcb-cluster-monitoring-operator-tls podName:56f88c5a-6833-4eab-b858-8c4b163fbdcb nodeName:}" failed. No retries permitted until 2026-07-07 15:01:09.727406978 +0000 UTC m=+37.523943702 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/56f88c5a-6833-4eab-b858-8c4b163fbdcb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-qt79g" (UID: "56f88c5a-6833-4eab-b858-8c4b163fbdcb") : secret "cluster-monitoring-operator-tls" not found
Jul 07 15:01:08.727590 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:08.727467    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/44e4a5b8-14bf-482c-9297-f2809ddb36c1-service-ca-bundle podName:44e4a5b8-14bf-482c-9297-f2809ddb36c1 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:09.727457833 +0000 UTC m=+37.523994538 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/44e4a5b8-14bf-482c-9297-f2809ddb36c1-service-ca-bundle") pod "router-default-58948c8cff-pclzr" (UID: "44e4a5b8-14bf-482c-9297-f2809ddb36c1") : configmap references non-existent config key: service-ca.crt
Jul 07 15:01:08.727590 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:08.727482    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 15:01:08.727590 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:08.727493    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cfbcb4769-87kk4: secret "image-registry-tls" not found
Jul 07 15:01:08.727590 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:08.727527    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-tls podName:d050a3c0-f99d-4e6d-99f3-03ce677e7fa0 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:09.727516138 +0000 UTC m=+37.524052858 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-tls") pod "image-registry-6cfbcb4769-87kk4" (UID: "d050a3c0-f99d-4e6d-99f3-03ce677e7fa0") : secret "image-registry-tls" not found
Jul 07 15:01:08.772388 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.772355    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:01:08.772388 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.772399    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:01:08.775620 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.775601    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 15:01:08.775734 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.775604    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ghr77\""
Jul 07 15:01:08.776113 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.775919    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bchjv\""
Jul 07 15:01:08.827963 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.827882    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-f4wfx\" (UID: \"9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f4wfx"
Jul 07 15:01:08.828105 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:08.828039    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 15:01:08.828144 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:08.828114    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118-samples-operator-tls podName:9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:09.82809587 +0000 UTC m=+37.624632580 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-f4wfx" (UID: "9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118") : secret "samples-operator-tls" not found
Jul 07 15:01:08.929217 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.929176    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23819f39-1440-4b6c-90fa-6db19eb48b74-metrics-tls\") pod \"dns-default-rlrhq\" (UID: \"23819f39-1440-4b6c-90fa-6db19eb48b74\") " pod="openshift-dns/dns-default-rlrhq"
Jul 07 15:01:08.929392 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:08.929339    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 15:01:08.929451 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:08.929406    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23819f39-1440-4b6c-90fa-6db19eb48b74-metrics-tls podName:23819f39-1440-4b6c-90fa-6db19eb48b74 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:09.929389053 +0000 UTC m=+37.725925759 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/23819f39-1440-4b6c-90fa-6db19eb48b74-metrics-tls") pod "dns-default-rlrhq" (UID: "23819f39-1440-4b6c-90fa-6db19eb48b74") : secret "dns-default-metrics-tls" not found
Jul 07 15:01:08.973487 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.973454    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-w9blj" event={"ID":"dbe1e84c-636f-4d7e-adcd-67aea6cd7a34","Type":"ContainerStarted","Data":"d77566911884e170dc45da831006a0690ce709d0f9fc0d39d50f73e01a9eb509"}
Jul 07 15:01:08.974465 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.974445    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-5r9rp" event={"ID":"1c51cd93-b0ea-4aa1-8678-a4c833b2d37e","Type":"ContainerStarted","Data":"5e3bf3648a940e8724d40eb7123d45cafade66ab9f70d4b77347685fb34d979e"}
Jul 07 15:01:08.975488 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.975467    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-rpzmw" event={"ID":"34299ca2-ae50-4512-a950-117fa8d0c013","Type":"ContainerStarted","Data":"438eb662d359957395bed9b1abfc69457182fb1df535332bdf84cb9ccc786bac"}
Jul 07 15:01:08.976445 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.976419    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ds29c" event={"ID":"f3ad7a15-08da-4441-8d40-2931adbefc20","Type":"ContainerStarted","Data":"b61ccb695c850a850ecdf368af68af8d37b5e557ec79a3cbe68c5dcb5cde33a7"}
Jul 07 15:01:08.977384 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.977367    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tfz7j" event={"ID":"719d2a5c-4213-4e2a-bce0-9b29e0ad02c0","Type":"ContainerStarted","Data":"55af4d438b4a45faedf12085872a48f3cab32ef0b2794d0ff8051f58eb70a7a7"}
Jul 07 15:01:08.978469 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.978451    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-cz626" event={"ID":"73d589b0-98d3-40b3-8179-27d4ff10efbf","Type":"ContainerStarted","Data":"5fc01550d62874c7d67205b884b706d8db7fc91b391280a7b25127186caa71c1"}
Jul 07 15:01:08.979395 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:08.979379    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-d5g6n" event={"ID":"422c8ccc-828c-4491-a88b-76cd6973e056","Type":"ContainerStarted","Data":"8c10f09b178c3b2d2e35547b3466cb054fa4a2d444e0a56472bdd8998a3d96c4"}
Jul 07 15:01:09.738948 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:09.738056    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-tls\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:09.738948 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:09.738115    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44e4a5b8-14bf-482c-9297-f2809ddb36c1-service-ca-bundle\") pod \"router-default-58948c8cff-pclzr\" (UID: \"44e4a5b8-14bf-482c-9297-f2809ddb36c1\") " pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:09.738948 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:09.738152    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44e4a5b8-14bf-482c-9297-f2809ddb36c1-metrics-certs\") pod \"router-default-58948c8cff-pclzr\" (UID: \"44e4a5b8-14bf-482c-9297-f2809ddb36c1\") " pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:09.738948 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:09.738269    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/56f88c5a-6833-4eab-b858-8c4b163fbdcb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-qt79g\" (UID: \"56f88c5a-6833-4eab-b858-8c4b163fbdcb\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qt79g"
Jul 07 15:01:09.738948 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:09.738413    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 15:01:09.738948 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:09.738482    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56f88c5a-6833-4eab-b858-8c4b163fbdcb-cluster-monitoring-operator-tls podName:56f88c5a-6833-4eab-b858-8c4b163fbdcb nodeName:}" failed. No retries permitted until 2026-07-07 15:01:11.738455696 +0000 UTC m=+39.534992407 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/56f88c5a-6833-4eab-b858-8c4b163fbdcb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-qt79g" (UID: "56f88c5a-6833-4eab-b858-8c4b163fbdcb") : secret "cluster-monitoring-operator-tls" not found
Jul 07 15:01:09.738948 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:09.738826    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/44e4a5b8-14bf-482c-9297-f2809ddb36c1-service-ca-bundle podName:44e4a5b8-14bf-482c-9297-f2809ddb36c1 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:11.738809242 +0000 UTC m=+39.535345955 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/44e4a5b8-14bf-482c-9297-f2809ddb36c1-service-ca-bundle") pod "router-default-58948c8cff-pclzr" (UID: "44e4a5b8-14bf-482c-9297-f2809ddb36c1") : configmap references non-existent config key: service-ca.crt
Jul 07 15:01:09.738948 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:09.738882    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 15:01:09.738948 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:09.738919    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44e4a5b8-14bf-482c-9297-f2809ddb36c1-metrics-certs podName:44e4a5b8-14bf-482c-9297-f2809ddb36c1 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:11.738907332 +0000 UTC m=+39.535444052 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/44e4a5b8-14bf-482c-9297-f2809ddb36c1-metrics-certs") pod "router-default-58948c8cff-pclzr" (UID: "44e4a5b8-14bf-482c-9297-f2809ddb36c1") : secret "router-metrics-certs-default" not found
Jul 07 15:01:09.738948 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:09.738919    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 15:01:09.738948 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:09.738933    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cfbcb4769-87kk4: secret "image-registry-tls" not found
Jul 07 15:01:09.738948 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:09.738961    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-tls podName:d050a3c0-f99d-4e6d-99f3-03ce677e7fa0 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:11.738952549 +0000 UTC m=+39.535489256 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-tls") pod "image-registry-6cfbcb4769-87kk4" (UID: "d050a3c0-f99d-4e6d-99f3-03ce677e7fa0") : secret "image-registry-tls" not found
Jul 07 15:01:09.771501 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:09.771468    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:01:09.776767 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:09.776268    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 15:01:09.839306 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:09.839267    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-f4wfx\" (UID: \"9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f4wfx"
Jul 07 15:01:09.839551 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:09.839532    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 15:01:09.839622 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:09.839599    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118-samples-operator-tls podName:9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:11.839580489 +0000 UTC m=+39.636117216 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-f4wfx" (UID: "9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118") : secret "samples-operator-tls" not found
Jul 07 15:01:09.940675 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:09.940637    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23819f39-1440-4b6c-90fa-6db19eb48b74-metrics-tls\") pod \"dns-default-rlrhq\" (UID: \"23819f39-1440-4b6c-90fa-6db19eb48b74\") " pod="openshift-dns/dns-default-rlrhq"
Jul 07 15:01:09.940923 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:09.940872    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 15:01:09.941021 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:09.940939    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23819f39-1440-4b6c-90fa-6db19eb48b74-metrics-tls podName:23819f39-1440-4b6c-90fa-6db19eb48b74 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:11.94091968 +0000 UTC m=+39.737456412 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/23819f39-1440-4b6c-90fa-6db19eb48b74-metrics-tls") pod "dns-default-rlrhq" (UID: "23819f39-1440-4b6c-90fa-6db19eb48b74") : secret "dns-default-metrics-tls" not found
Jul 07 15:01:11.007655 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:11.006691    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ds29c" event={"ID":"f3ad7a15-08da-4441-8d40-2931adbefc20","Type":"ContainerStarted","Data":"351797b53f2dd3de40f35b08c08d7311394d4001152eac28238c7ad634dd6a8b"}
Jul 07 15:01:11.026766 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:11.025267    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ds29c" podStartSLOduration=10.089370009 podStartE2EDuration="12.02524513s" totalImagesPullingTime="1.935875121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:00:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:11.024957265 +0000 UTC m=+38.821494015" watchObservedRunningTime="2026-07-07 15:01:11.02524513 +0000 UTC m=+38.821781861"
Jul 07 15:01:11.164270 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:11.164219    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tg7l_671cc2c7-4b8d-4183-8a74-16d971f3431e/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:11.295730 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:11.295597    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tg7l_671cc2c7-4b8d-4183-8a74-16d971f3431e/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:11.393596 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:11.393511    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tg7l_671cc2c7-4b8d-4183-8a74-16d971f3431e/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:11.495096 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:11.495055    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tg7l_671cc2c7-4b8d-4183-8a74-16d971f3431e/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:11.561103 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:11.561043    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tg7l_671cc2c7-4b8d-4183-8a74-16d971f3431e/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:11.763776 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:11.763738    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44e4a5b8-14bf-482c-9297-f2809ddb36c1-metrics-certs\") pod \"router-default-58948c8cff-pclzr\" (UID: \"44e4a5b8-14bf-482c-9297-f2809ddb36c1\") " pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:11.763965 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:11.763898    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/56f88c5a-6833-4eab-b858-8c4b163fbdcb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-qt79g\" (UID: \"56f88c5a-6833-4eab-b858-8c4b163fbdcb\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qt79g"
Jul 07 15:01:11.764025 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:11.763968    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-tls\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:11.764025 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:11.764008    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44e4a5b8-14bf-482c-9297-f2809ddb36c1-service-ca-bundle\") pod \"router-default-58948c8cff-pclzr\" (UID: \"44e4a5b8-14bf-482c-9297-f2809ddb36c1\") " pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:11.764201 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:11.764174    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/44e4a5b8-14bf-482c-9297-f2809ddb36c1-service-ca-bundle podName:44e4a5b8-14bf-482c-9297-f2809ddb36c1 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:15.764152533 +0000 UTC m=+43.560689242 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/44e4a5b8-14bf-482c-9297-f2809ddb36c1-service-ca-bundle") pod "router-default-58948c8cff-pclzr" (UID: "44e4a5b8-14bf-482c-9297-f2809ddb36c1") : configmap references non-existent config key: service-ca.crt
Jul 07 15:01:11.764644 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:11.764618    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 15:01:11.764732 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:11.764682    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44e4a5b8-14bf-482c-9297-f2809ddb36c1-metrics-certs podName:44e4a5b8-14bf-482c-9297-f2809ddb36c1 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:15.764663965 +0000 UTC m=+43.561200674 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/44e4a5b8-14bf-482c-9297-f2809ddb36c1-metrics-certs") pod "router-default-58948c8cff-pclzr" (UID: "44e4a5b8-14bf-482c-9297-f2809ddb36c1") : secret "router-metrics-certs-default" not found
Jul 07 15:01:11.764956 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:11.764927    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 15:01:11.764956 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:11.764946    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cfbcb4769-87kk4: secret "image-registry-tls" not found
Jul 07 15:01:11.765134 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:11.765011    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-tls podName:d050a3c0-f99d-4e6d-99f3-03ce677e7fa0 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:15.764994727 +0000 UTC m=+43.561531438 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-tls") pod "image-registry-6cfbcb4769-87kk4" (UID: "d050a3c0-f99d-4e6d-99f3-03ce677e7fa0") : secret "image-registry-tls" not found
Jul 07 15:01:11.765134 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:11.765101    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 15:01:11.765278 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:11.765172    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56f88c5a-6833-4eab-b858-8c4b163fbdcb-cluster-monitoring-operator-tls podName:56f88c5a-6833-4eab-b858-8c4b163fbdcb nodeName:}" failed. No retries permitted until 2026-07-07 15:01:15.765158005 +0000 UTC m=+43.561694727 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/56f88c5a-6833-4eab-b858-8c4b163fbdcb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-qt79g" (UID: "56f88c5a-6833-4eab-b858-8c4b163fbdcb") : secret "cluster-monitoring-operator-tls" not found
Jul 07 15:01:11.864744 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:11.864657    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-f4wfx\" (UID: \"9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f4wfx"
Jul 07 15:01:11.864954 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:11.864805    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 15:01:11.864954 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:11.864894    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118-samples-operator-tls podName:9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:15.86487634 +0000 UTC m=+43.661413051 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-f4wfx" (UID: "9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118") : secret "samples-operator-tls" not found
Jul 07 15:01:11.966422 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:11.965579    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23819f39-1440-4b6c-90fa-6db19eb48b74-metrics-tls\") pod \"dns-default-rlrhq\" (UID: \"23819f39-1440-4b6c-90fa-6db19eb48b74\") " pod="openshift-dns/dns-default-rlrhq"
Jul 07 15:01:11.966422 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:11.965942    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 15:01:11.966422 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:11.966022    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23819f39-1440-4b6c-90fa-6db19eb48b74-metrics-tls podName:23819f39-1440-4b6c-90fa-6db19eb48b74 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:15.966000209 +0000 UTC m=+43.762536921 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/23819f39-1440-4b6c-90fa-6db19eb48b74-metrics-tls") pod "dns-default-rlrhq" (UID: "23819f39-1440-4b6c-90fa-6db19eb48b74") : secret "dns-default-metrics-tls" not found
Jul 07 15:01:12.219336 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:12.219301    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tg7l_671cc2c7-4b8d-4183-8a74-16d971f3431e/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:12.279366 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:12.279336    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tg7l_671cc2c7-4b8d-4183-8a74-16d971f3431e/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:12.616315 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:12.616283    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tg7l_671cc2c7-4b8d-4183-8a74-16d971f3431e/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:12.677406 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:12.677369    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tg7l_671cc2c7-4b8d-4183-8a74-16d971f3431e/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:13.090797 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:13.090758    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tg7l_671cc2c7-4b8d-4183-8a74-16d971f3431e/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:13.329021 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:13.328982    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7tg7l_671cc2c7-4b8d-4183-8a74-16d971f3431e/konnectivity-agent/0.log" limit=1024
Jul 07 15:01:15.795995 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:15.795950    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/56f88c5a-6833-4eab-b858-8c4b163fbdcb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-qt79g\" (UID: \"56f88c5a-6833-4eab-b858-8c4b163fbdcb\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qt79g"
Jul 07 15:01:15.796621 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:15.796017    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-tls\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:15.796621 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:15.796047    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44e4a5b8-14bf-482c-9297-f2809ddb36c1-service-ca-bundle\") pod \"router-default-58948c8cff-pclzr\" (UID: \"44e4a5b8-14bf-482c-9297-f2809ddb36c1\") " pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:15.796621 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:15.796077    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44e4a5b8-14bf-482c-9297-f2809ddb36c1-metrics-certs\") pod \"router-default-58948c8cff-pclzr\" (UID: \"44e4a5b8-14bf-482c-9297-f2809ddb36c1\") " pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:15.796621 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:15.796127    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 15:01:15.796621 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:15.796199    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 15:01:15.796621 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:15.796206    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 15:01:15.796621 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:15.796227    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cfbcb4769-87kk4: secret "image-registry-tls" not found
Jul 07 15:01:15.796621 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:15.796271    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/44e4a5b8-14bf-482c-9297-f2809ddb36c1-service-ca-bundle podName:44e4a5b8-14bf-482c-9297-f2809ddb36c1 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:23.796245823 +0000 UTC m=+51.592782534 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/44e4a5b8-14bf-482c-9297-f2809ddb36c1-service-ca-bundle") pod "router-default-58948c8cff-pclzr" (UID: "44e4a5b8-14bf-482c-9297-f2809ddb36c1") : configmap references non-existent config key: service-ca.crt
Jul 07 15:01:15.796621 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:15.796298    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44e4a5b8-14bf-482c-9297-f2809ddb36c1-metrics-certs podName:44e4a5b8-14bf-482c-9297-f2809ddb36c1 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:23.796287455 +0000 UTC m=+51.592824162 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/44e4a5b8-14bf-482c-9297-f2809ddb36c1-metrics-certs") pod "router-default-58948c8cff-pclzr" (UID: "44e4a5b8-14bf-482c-9297-f2809ddb36c1") : secret "router-metrics-certs-default" not found
Jul 07 15:01:15.796621 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:15.796312    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-tls podName:d050a3c0-f99d-4e6d-99f3-03ce677e7fa0 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:23.796304071 +0000 UTC m=+51.592840777 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-tls") pod "image-registry-6cfbcb4769-87kk4" (UID: "d050a3c0-f99d-4e6d-99f3-03ce677e7fa0") : secret "image-registry-tls" not found
Jul 07 15:01:15.796621 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:15.796325    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56f88c5a-6833-4eab-b858-8c4b163fbdcb-cluster-monitoring-operator-tls podName:56f88c5a-6833-4eab-b858-8c4b163fbdcb nodeName:}" failed. No retries permitted until 2026-07-07 15:01:23.796317585 +0000 UTC m=+51.592854310 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/56f88c5a-6833-4eab-b858-8c4b163fbdcb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-qt79g" (UID: "56f88c5a-6833-4eab-b858-8c4b163fbdcb") : secret "cluster-monitoring-operator-tls" not found
Jul 07 15:01:15.897359 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:15.897324    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-f4wfx\" (UID: \"9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f4wfx"
Jul 07 15:01:15.897533 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:15.897479    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 15:01:15.897644 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:15.897553    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118-samples-operator-tls podName:9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:23.897532092 +0000 UTC m=+51.694068819 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-f4wfx" (UID: "9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118") : secret "samples-operator-tls" not found
Jul 07 15:01:15.998556 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:15.998523    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23819f39-1440-4b6c-90fa-6db19eb48b74-metrics-tls\") pod \"dns-default-rlrhq\" (UID: \"23819f39-1440-4b6c-90fa-6db19eb48b74\") " pod="openshift-dns/dns-default-rlrhq"
Jul 07 15:01:15.998731 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:15.998695    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 15:01:15.998793 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:15.998787    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23819f39-1440-4b6c-90fa-6db19eb48b74-metrics-tls podName:23819f39-1440-4b6c-90fa-6db19eb48b74 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:23.998766241 +0000 UTC m=+51.795302958 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/23819f39-1440-4b6c-90fa-6db19eb48b74-metrics-tls") pod "dns-default-rlrhq" (UID: "23819f39-1440-4b6c-90fa-6db19eb48b74") : secret "dns-default-metrics-tls" not found
Jul 07 15:01:19.035938 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:19.035917    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tfz7j" event={"ID":"719d2a5c-4213-4e2a-bce0-9b29e0ad02c0","Type":"ContainerStarted","Data":"22d761a1dc7acad0de92aea7bd050ef2da017c002aee52b238cc5da074c0788a"}
Jul 07 15:01:19.038729 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:19.038273    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-cz626" event={"ID":"73d589b0-98d3-40b3-8179-27d4ff10efbf","Type":"ContainerStarted","Data":"ed859ca60631a73c08aeb86df55345f8b12987bfa3ef4025944b8395bd9e5c0e"}
Jul 07 15:01:19.042331 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:19.041920    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-d5g6n" event={"ID":"422c8ccc-828c-4491-a88b-76cd6973e056","Type":"ContainerStarted","Data":"34b537d54a79c26ed1da1691eb062019183657f19040914cd751d6bf4ed7b0a8"}
Jul 07 15:01:19.044585 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:19.044173    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-w9blj" event={"ID":"dbe1e84c-636f-4d7e-adcd-67aea6cd7a34","Type":"ContainerStarted","Data":"4dec59c72d7145610436b1a05c1a45dc27ef69b9aafd8cd32021b5255c0c0697"}
Jul 07 15:01:19.048635 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:19.046952    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-5r9rp" event={"ID":"1c51cd93-b0ea-4aa1-8678-a4c833b2d37e","Type":"ContainerStarted","Data":"d60b2090d8da672a652cf239e41e5140d89ce0527cc5dd49d844cea201e1a6f6"}
Jul 07 15:01:19.049008 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:19.048802    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-5r9rp"
Jul 07 15:01:19.049008 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:19.048916    2567 patch_prober.go:28] interesting pod/console-operator-68b757fb67-5r9rp container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.14:8443/readyz\": dial tcp 10.132.0.14:8443: connect: connection refused" start-of-body=
Jul 07 15:01:19.049008 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:19.048952    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-68b757fb67-5r9rp" podUID="1c51cd93-b0ea-4aa1-8678-a4c833b2d37e" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.14:8443/readyz\": dial tcp 10.132.0.14:8443: connect: connection refused"
Jul 07 15:01:19.094695 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:19.094470    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tfz7j" podStartSLOduration=5.03945904 podStartE2EDuration="15.094446901s" totalImagesPullingTime="10.054987861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:19.093216481 +0000 UTC m=+46.889753238" watchObservedRunningTime="2026-07-07 15:01:19.094446901 +0000 UTC m=+46.890983630"
Jul 07 15:01:19.122671 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:19.122599    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-cz626" podStartSLOduration=4.076521112 podStartE2EDuration="14.122577525s" totalImagesPullingTime="10.046056413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:19.121295639 +0000 UTC m=+46.917832368" watchObservedRunningTime="2026-07-07 15:01:19.122577525 +0000 UTC m=+46.919114254"
Jul 07 15:01:19.186598 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:19.186537    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-rpzmw" podStartSLOduration=2.162080485 podStartE2EDuration="12.186513902s" totalImagesPullingTime="10.024433417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:19.156946195 +0000 UTC m=+46.953482941" watchObservedRunningTime="2026-07-07 15:01:19.186513902 +0000 UTC m=+46.983050630"
Jul 07 15:01:19.212809 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:19.212761    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-d5g6n" podStartSLOduration=14.175467086 podStartE2EDuration="24.212743582s" totalImagesPullingTime="10.037276496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:00:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:19.187003083 +0000 UTC m=+46.983539810" watchObservedRunningTime="2026-07-07 15:01:19.212743582 +0000 UTC m=+47.009280311"
Jul 07 15:01:19.238682 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:19.238624    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-w9blj" podStartSLOduration=1.2348790570000001 podStartE2EDuration="11.23860218s" totalImagesPullingTime="10.003723123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:19.237145501 +0000 UTC m=+47.033682230" watchObservedRunningTime="2026-07-07 15:01:19.23860218 +0000 UTC m=+47.035138909"
Jul 07 15:01:19.239305 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:19.239269    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-5r9rp" podStartSLOduration=9.22203078 podStartE2EDuration="19.239257668s" totalImagesPullingTime="10.017226888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:19.212112854 +0000 UTC m=+47.008649586" watchObservedRunningTime="2026-07-07 15:01:19.239257668 +0000 UTC m=+47.035794398"
Jul 07 15:01:20.056119 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:20.056079    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-rpzmw" event={"ID":"34299ca2-ae50-4512-a950-117fa8d0c013","Type":"ContainerStarted","Data":"5825af3435039977279357d53384e35706caf81d6e6741b1a9d58f6088163cdc"}
Jul 07 15:01:20.059207 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:20.059179    2567 generic.go:356] "Generic (PLEG): container finished" podID="c6eef884-173e-4c5b-890d-3409f42fb0a3" containerID="c1ba5f8784531301b07da45c257f99a0664caca7630dc6ca5ef2a1ffeb331599" exitCode=0
Jul 07 15:01:20.059351 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:20.059264    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5r27h" event={"ID":"c6eef884-173e-4c5b-890d-3409f42fb0a3","Type":"ContainerDied","Data":"c1ba5f8784531301b07da45c257f99a0664caca7630dc6ca5ef2a1ffeb331599"}
Jul 07 15:01:20.061053 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:20.061030    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5r9rp_1c51cd93-b0ea-4aa1-8678-a4c833b2d37e/console-operator/0.log"
Jul 07 15:01:20.061156 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:20.061071    2567 generic.go:356] "Generic (PLEG): container finished" podID="1c51cd93-b0ea-4aa1-8678-a4c833b2d37e" containerID="d60b2090d8da672a652cf239e41e5140d89ce0527cc5dd49d844cea201e1a6f6" exitCode=255
Jul 07 15:01:20.061212 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:20.061155    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-5r9rp" event={"ID":"1c51cd93-b0ea-4aa1-8678-a4c833b2d37e","Type":"ContainerDied","Data":"d60b2090d8da672a652cf239e41e5140d89ce0527cc5dd49d844cea201e1a6f6"}
Jul 07 15:01:20.061370 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:20.061353    2567 scope.go:117] "RemoveContainer" containerID="d60b2090d8da672a652cf239e41e5140d89ce0527cc5dd49d844cea201e1a6f6"
Jul 07 15:01:20.851824 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:20.851790    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6m26d"]
Jul 07 15:01:20.868616 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:20.868581    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6m26d"
Jul 07 15:01:20.871994 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:20.871959    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6m26d"]
Jul 07 15:01:20.874916 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:20.874893    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4zds6\""
Jul 07 15:01:20.875306 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:20.875289    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 07 15:01:20.875443 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:20.875340    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 07 15:01:20.952142 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:20.952104    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c81a8d96-38ae-48b5-bd30-2711751dedef-crio-socket\") pod \"insights-runtime-extractor-6m26d\" (UID: \"c81a8d96-38ae-48b5-bd30-2711751dedef\") " pod="openshift-insights/insights-runtime-extractor-6m26d"
Jul 07 15:01:20.952142 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:20.952143    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c81a8d96-38ae-48b5-bd30-2711751dedef-data-volume\") pod \"insights-runtime-extractor-6m26d\" (UID: \"c81a8d96-38ae-48b5-bd30-2711751dedef\") " pod="openshift-insights/insights-runtime-extractor-6m26d"
Jul 07 15:01:20.952337 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:20.952186    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c81a8d96-38ae-48b5-bd30-2711751dedef-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6m26d\" (UID: \"c81a8d96-38ae-48b5-bd30-2711751dedef\") " pod="openshift-insights/insights-runtime-extractor-6m26d"
Jul 07 15:01:20.952337 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:20.952254    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c81a8d96-38ae-48b5-bd30-2711751dedef-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6m26d\" (UID: \"c81a8d96-38ae-48b5-bd30-2711751dedef\") " pod="openshift-insights/insights-runtime-extractor-6m26d"
Jul 07 15:01:20.952414 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:20.952356    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss42z\" (UniqueName: \"kubernetes.io/projected/c81a8d96-38ae-48b5-bd30-2711751dedef-kube-api-access-ss42z\") pod \"insights-runtime-extractor-6m26d\" (UID: \"c81a8d96-38ae-48b5-bd30-2711751dedef\") " pod="openshift-insights/insights-runtime-extractor-6m26d"
Jul 07 15:01:21.052943 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:21.052902    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c81a8d96-38ae-48b5-bd30-2711751dedef-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6m26d\" (UID: \"c81a8d96-38ae-48b5-bd30-2711751dedef\") " pod="openshift-insights/insights-runtime-extractor-6m26d"
Jul 07 15:01:21.053111 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:21.052965    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c81a8d96-38ae-48b5-bd30-2711751dedef-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6m26d\" (UID: \"c81a8d96-38ae-48b5-bd30-2711751dedef\") " pod="openshift-insights/insights-runtime-extractor-6m26d"
Jul 07 15:01:21.053111 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:21.053037    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 15:01:21.053111 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:21.053054    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss42z\" (UniqueName: \"kubernetes.io/projected/c81a8d96-38ae-48b5-bd30-2711751dedef-kube-api-access-ss42z\") pod \"insights-runtime-extractor-6m26d\" (UID: \"c81a8d96-38ae-48b5-bd30-2711751dedef\") " pod="openshift-insights/insights-runtime-extractor-6m26d"
Jul 07 15:01:21.053111 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:21.053107    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c81a8d96-38ae-48b5-bd30-2711751dedef-kube-rbac-proxy-cm podName:c81a8d96-38ae-48b5-bd30-2711751dedef nodeName:}" failed. No retries permitted until 2026-07-07 15:01:21.553087991 +0000 UTC m=+49.349624714 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c81a8d96-38ae-48b5-bd30-2711751dedef-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6m26d" (UID: "c81a8d96-38ae-48b5-bd30-2711751dedef") : configmap "kube-rbac-proxy" not found
Jul 07 15:01:21.053283 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:21.053147    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c81a8d96-38ae-48b5-bd30-2711751dedef-crio-socket\") pod \"insights-runtime-extractor-6m26d\" (UID: \"c81a8d96-38ae-48b5-bd30-2711751dedef\") " pod="openshift-insights/insights-runtime-extractor-6m26d"
Jul 07 15:01:21.053283 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:21.053166    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 15:01:21.053283 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:21.053179    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c81a8d96-38ae-48b5-bd30-2711751dedef-data-volume\") pod \"insights-runtime-extractor-6m26d\" (UID: \"c81a8d96-38ae-48b5-bd30-2711751dedef\") " pod="openshift-insights/insights-runtime-extractor-6m26d"
Jul 07 15:01:21.053283 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:21.053249    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c81a8d96-38ae-48b5-bd30-2711751dedef-insights-runtime-extractor-tls podName:c81a8d96-38ae-48b5-bd30-2711751dedef nodeName:}" failed. No retries permitted until 2026-07-07 15:01:21.553229765 +0000 UTC m=+49.349766491 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c81a8d96-38ae-48b5-bd30-2711751dedef-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6m26d" (UID: "c81a8d96-38ae-48b5-bd30-2711751dedef") : secret "insights-runtime-extractor-tls" not found
Jul 07 15:01:21.053472 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:21.053438    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c81a8d96-38ae-48b5-bd30-2711751dedef-crio-socket\") pod \"insights-runtime-extractor-6m26d\" (UID: \"c81a8d96-38ae-48b5-bd30-2711751dedef\") " pod="openshift-insights/insights-runtime-extractor-6m26d"
Jul 07 15:01:21.053512 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:21.053483    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c81a8d96-38ae-48b5-bd30-2711751dedef-data-volume\") pod \"insights-runtime-extractor-6m26d\" (UID: \"c81a8d96-38ae-48b5-bd30-2711751dedef\") " pod="openshift-insights/insights-runtime-extractor-6m26d"
Jul 07 15:01:21.065554 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:21.065517    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss42z\" (UniqueName: \"kubernetes.io/projected/c81a8d96-38ae-48b5-bd30-2711751dedef-kube-api-access-ss42z\") pod \"insights-runtime-extractor-6m26d\" (UID: \"c81a8d96-38ae-48b5-bd30-2711751dedef\") " pod="openshift-insights/insights-runtime-extractor-6m26d"
Jul 07 15:01:21.069091 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:21.069059    2567 generic.go:356] "Generic (PLEG): container finished" podID="c6eef884-173e-4c5b-890d-3409f42fb0a3" containerID="019d6c38ca8b09d93cddb19de806c9a227405d93b3f74c03abd07f55532bded3" exitCode=0
Jul 07 15:01:21.069248 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:21.069138    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5r27h" event={"ID":"c6eef884-173e-4c5b-890d-3409f42fb0a3","Type":"ContainerDied","Data":"019d6c38ca8b09d93cddb19de806c9a227405d93b3f74c03abd07f55532bded3"}
Jul 07 15:01:21.071687 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:21.071668    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5r9rp_1c51cd93-b0ea-4aa1-8678-a4c833b2d37e/console-operator/1.log"
Jul 07 15:01:21.072161 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:21.072148    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5r9rp_1c51cd93-b0ea-4aa1-8678-a4c833b2d37e/console-operator/0.log"
Jul 07 15:01:21.072230 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:21.072185    2567 generic.go:356] "Generic (PLEG): container finished" podID="1c51cd93-b0ea-4aa1-8678-a4c833b2d37e" containerID="1e4b1ea7047fd6028f958306e7ff1d8faae388a16124e236b9526c396317efb2" exitCode=255
Jul 07 15:01:21.072282 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:21.072247    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-5r9rp" event={"ID":"1c51cd93-b0ea-4aa1-8678-a4c833b2d37e","Type":"ContainerDied","Data":"1e4b1ea7047fd6028f958306e7ff1d8faae388a16124e236b9526c396317efb2"}
Jul 07 15:01:21.072330 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:21.072299    2567 scope.go:117] "RemoveContainer" containerID="d60b2090d8da672a652cf239e41e5140d89ce0527cc5dd49d844cea201e1a6f6"
Jul 07 15:01:21.072481 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:21.072453    2567 scope.go:117] "RemoveContainer" containerID="1e4b1ea7047fd6028f958306e7ff1d8faae388a16124e236b9526c396317efb2"
Jul 07 15:01:21.072648 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:21.072622    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-5r9rp_openshift-console-operator(1c51cd93-b0ea-4aa1-8678-a4c833b2d37e)\"" pod="openshift-console-operator/console-operator-68b757fb67-5r9rp" podUID="1c51cd93-b0ea-4aa1-8678-a4c833b2d37e"
Jul 07 15:01:21.556694 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:21.556663    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c81a8d96-38ae-48b5-bd30-2711751dedef-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6m26d\" (UID: \"c81a8d96-38ae-48b5-bd30-2711751dedef\") " pod="openshift-insights/insights-runtime-extractor-6m26d"
Jul 07 15:01:21.556860 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:21.556727    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c81a8d96-38ae-48b5-bd30-2711751dedef-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6m26d\" (UID: \"c81a8d96-38ae-48b5-bd30-2711751dedef\") " pod="openshift-insights/insights-runtime-extractor-6m26d"
Jul 07 15:01:21.556860 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:21.556786    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 15:01:21.556860 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:21.556820    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 15:01:21.556977 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:21.556873    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c81a8d96-38ae-48b5-bd30-2711751dedef-kube-rbac-proxy-cm podName:c81a8d96-38ae-48b5-bd30-2711751dedef nodeName:}" failed. No retries permitted until 2026-07-07 15:01:22.556855515 +0000 UTC m=+50.353392223 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c81a8d96-38ae-48b5-bd30-2711751dedef-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6m26d" (UID: "c81a8d96-38ae-48b5-bd30-2711751dedef") : configmap "kube-rbac-proxy" not found
Jul 07 15:01:21.556977 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:21.556892    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c81a8d96-38ae-48b5-bd30-2711751dedef-insights-runtime-extractor-tls podName:c81a8d96-38ae-48b5-bd30-2711751dedef nodeName:}" failed. No retries permitted until 2026-07-07 15:01:22.556880257 +0000 UTC m=+50.353416976 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c81a8d96-38ae-48b5-bd30-2711751dedef-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6m26d" (UID: "c81a8d96-38ae-48b5-bd30-2711751dedef") : secret "insights-runtime-extractor-tls" not found
Jul 07 15:01:22.076154 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:22.076120    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-d2br4_4017287a-0129-4a52-8875-045408b9a82d/dns-node-resolver/0.log"
Jul 07 15:01:22.079318 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:22.079285    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5r27h" event={"ID":"c6eef884-173e-4c5b-890d-3409f42fb0a3","Type":"ContainerStarted","Data":"ff261f08fabd8ed7b09171c9f3eee718a54829810b8fff8fb92232a181334a44"}
Jul 07 15:01:22.080672 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:22.080654    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5r9rp_1c51cd93-b0ea-4aa1-8678-a4c833b2d37e/console-operator/1.log"
Jul 07 15:01:22.080957 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:22.080944    2567 scope.go:117] "RemoveContainer" containerID="1e4b1ea7047fd6028f958306e7ff1d8faae388a16124e236b9526c396317efb2"
Jul 07 15:01:22.081105 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:22.081088    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-5r9rp_openshift-console-operator(1c51cd93-b0ea-4aa1-8678-a4c833b2d37e)\"" pod="openshift-console-operator/console-operator-68b757fb67-5r9rp" podUID="1c51cd93-b0ea-4aa1-8678-a4c833b2d37e"
Jul 07 15:01:22.109029 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:22.108976    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5r27h" podStartSLOduration=7.860791945 podStartE2EDuration="49.108958158s" totalImagesPullingTime="41.248166213s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:00:33 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:22.106741775 +0000 UTC m=+49.903278494" watchObservedRunningTime="2026-07-07 15:01:22.108958158 +0000 UTC m=+49.905494887"
Jul 07 15:01:22.252703 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:22.252674    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-42n8j"]
Jul 07 15:01:22.262188 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:22.262164    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-42n8j"
Jul 07 15:01:22.265288 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:22.265270    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 07 15:01:22.265624 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:22.265604    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 07 15:01:22.266959 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:22.266942    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 07 15:01:22.267079 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:22.266994    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 07 15:01:22.267079 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:22.267056    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-5twp8\""
Jul 07 15:01:22.268419 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:22.268397    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-42n8j"]
Jul 07 15:01:22.365282 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:22.365193    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58wfw\" (UniqueName: \"kubernetes.io/projected/63788c0a-72f9-4ec8-9363-c3063772bb95-kube-api-access-58wfw\") pod \"service-ca-758f599c5-42n8j\" (UID: \"63788c0a-72f9-4ec8-9363-c3063772bb95\") " pod="openshift-service-ca/service-ca-758f599c5-42n8j"
Jul 07 15:01:22.365427 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:22.365290    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/63788c0a-72f9-4ec8-9363-c3063772bb95-signing-key\") pod \"service-ca-758f599c5-42n8j\" (UID: \"63788c0a-72f9-4ec8-9363-c3063772bb95\") " pod="openshift-service-ca/service-ca-758f599c5-42n8j"
Jul 07 15:01:22.365427 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:22.365390    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/63788c0a-72f9-4ec8-9363-c3063772bb95-signing-cabundle\") pod \"service-ca-758f599c5-42n8j\" (UID: \"63788c0a-72f9-4ec8-9363-c3063772bb95\") " pod="openshift-service-ca/service-ca-758f599c5-42n8j"
Jul 07 15:01:22.466792 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:22.466754    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58wfw\" (UniqueName: \"kubernetes.io/projected/63788c0a-72f9-4ec8-9363-c3063772bb95-kube-api-access-58wfw\") pod \"service-ca-758f599c5-42n8j\" (UID: \"63788c0a-72f9-4ec8-9363-c3063772bb95\") " pod="openshift-service-ca/service-ca-758f599c5-42n8j"
Jul 07 15:01:22.466982 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:22.466821    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/63788c0a-72f9-4ec8-9363-c3063772bb95-signing-key\") pod \"service-ca-758f599c5-42n8j\" (UID: \"63788c0a-72f9-4ec8-9363-c3063772bb95\") " pod="openshift-service-ca/service-ca-758f599c5-42n8j"
Jul 07 15:01:22.467064 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:22.466984    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/63788c0a-72f9-4ec8-9363-c3063772bb95-signing-cabundle\") pod \"service-ca-758f599c5-42n8j\" (UID: \"63788c0a-72f9-4ec8-9363-c3063772bb95\") " pod="openshift-service-ca/service-ca-758f599c5-42n8j"
Jul 07 15:01:22.467612 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:22.467587    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/63788c0a-72f9-4ec8-9363-c3063772bb95-signing-cabundle\") pod \"service-ca-758f599c5-42n8j\" (UID: \"63788c0a-72f9-4ec8-9363-c3063772bb95\") " pod="openshift-service-ca/service-ca-758f599c5-42n8j"
Jul 07 15:01:22.469448 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:22.469426    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/63788c0a-72f9-4ec8-9363-c3063772bb95-signing-key\") pod \"service-ca-758f599c5-42n8j\" (UID: \"63788c0a-72f9-4ec8-9363-c3063772bb95\") " pod="openshift-service-ca/service-ca-758f599c5-42n8j"
Jul 07 15:01:22.476719 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:22.476697    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58wfw\" (UniqueName: \"kubernetes.io/projected/63788c0a-72f9-4ec8-9363-c3063772bb95-kube-api-access-58wfw\") pod \"service-ca-758f599c5-42n8j\" (UID: \"63788c0a-72f9-4ec8-9363-c3063772bb95\") " pod="openshift-service-ca/service-ca-758f599c5-42n8j"
Jul 07 15:01:22.568372 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:22.568315    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c81a8d96-38ae-48b5-bd30-2711751dedef-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6m26d\" (UID: \"c81a8d96-38ae-48b5-bd30-2711751dedef\") " pod="openshift-insights/insights-runtime-extractor-6m26d"
Jul 07 15:01:22.568583 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:22.568384    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c81a8d96-38ae-48b5-bd30-2711751dedef-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6m26d\" (UID: \"c81a8d96-38ae-48b5-bd30-2711751dedef\") " pod="openshift-insights/insights-runtime-extractor-6m26d"
Jul 07 15:01:22.568583 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:22.568452    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 15:01:22.568583 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:22.568486    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 15:01:22.568583 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:22.568533    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c81a8d96-38ae-48b5-bd30-2711751dedef-kube-rbac-proxy-cm podName:c81a8d96-38ae-48b5-bd30-2711751dedef nodeName:}" failed. No retries permitted until 2026-07-07 15:01:24.568512278 +0000 UTC m=+52.365049001 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c81a8d96-38ae-48b5-bd30-2711751dedef-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6m26d" (UID: "c81a8d96-38ae-48b5-bd30-2711751dedef") : configmap "kube-rbac-proxy" not found
Jul 07 15:01:22.568583 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:22.568553    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c81a8d96-38ae-48b5-bd30-2711751dedef-insights-runtime-extractor-tls podName:c81a8d96-38ae-48b5-bd30-2711751dedef nodeName:}" failed. No retries permitted until 2026-07-07 15:01:24.568544819 +0000 UTC m=+52.365081528 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c81a8d96-38ae-48b5-bd30-2711751dedef-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6m26d" (UID: "c81a8d96-38ae-48b5-bd30-2711751dedef") : secret "insights-runtime-extractor-tls" not found
Jul 07 15:01:22.572342 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:22.572316    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-42n8j"
Jul 07 15:01:22.726286 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:22.726252    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-42n8j"]
Jul 07 15:01:22.743071 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:01:22.743040    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63788c0a_72f9_4ec8_9363_c3063772bb95.slice/crio-37c2d8e93e0f9064abe3c94664b22dd87bb821b769329db732f79739e426478e WatchSource:0}: Error finding container 37c2d8e93e0f9064abe3c94664b22dd87bb821b769329db732f79739e426478e: Status 404 returned error can't find the container with id 37c2d8e93e0f9064abe3c94664b22dd87bb821b769329db732f79739e426478e
Jul 07 15:01:23.085264 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:23.085225    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-42n8j" event={"ID":"63788c0a-72f9-4ec8-9363-c3063772bb95","Type":"ContainerStarted","Data":"8d6e32ea5e8de108f038124c488f051ddee770cdfdeeeb319919185e0bc086e4"}
Jul 07 15:01:23.085735 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:23.085275    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-42n8j" event={"ID":"63788c0a-72f9-4ec8-9363-c3063772bb95","Type":"ContainerStarted","Data":"37c2d8e93e0f9064abe3c94664b22dd87bb821b769329db732f79739e426478e"}
Jul 07 15:01:23.110226 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:23.110175    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-42n8j" podStartSLOduration=1.110158597 podStartE2EDuration="1.110158597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:23.1093771 +0000 UTC m=+50.905913849" watchObservedRunningTime="2026-07-07 15:01:23.110158597 +0000 UTC m=+50.906695325"
Jul 07 15:01:23.276723 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:23.276698    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zk8hk_88f0b024-94cb-44b4-a5f4-9497ce96c3cb/node-ca/0.log"
Jul 07 15:01:23.690043 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:23.690007    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-w9blj_dbe1e84c-636f-4d7e-adcd-67aea6cd7a34/serve-healthcheck-canary/0.log"
Jul 07 15:01:23.879509 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:23.879461    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44e4a5b8-14bf-482c-9297-f2809ddb36c1-metrics-certs\") pod \"router-default-58948c8cff-pclzr\" (UID: \"44e4a5b8-14bf-482c-9297-f2809ddb36c1\") " pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:23.879701 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:23.879585    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/56f88c5a-6833-4eab-b858-8c4b163fbdcb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-qt79g\" (UID: \"56f88c5a-6833-4eab-b858-8c4b163fbdcb\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qt79g"
Jul 07 15:01:23.879701 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:23.879653    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-tls\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:23.879701 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:23.879696    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44e4a5b8-14bf-482c-9297-f2809ddb36c1-service-ca-bundle\") pod \"router-default-58948c8cff-pclzr\" (UID: \"44e4a5b8-14bf-482c-9297-f2809ddb36c1\") " pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:23.879874 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:23.879859    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/44e4a5b8-14bf-482c-9297-f2809ddb36c1-service-ca-bundle podName:44e4a5b8-14bf-482c-9297-f2809ddb36c1 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:39.879823486 +0000 UTC m=+67.676360197 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/44e4a5b8-14bf-482c-9297-f2809ddb36c1-service-ca-bundle") pod "router-default-58948c8cff-pclzr" (UID: "44e4a5b8-14bf-482c-9297-f2809ddb36c1") : configmap references non-existent config key: service-ca.crt
Jul 07 15:01:23.880092 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:23.880072    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 15:01:23.880252 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:23.880233    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56f88c5a-6833-4eab-b858-8c4b163fbdcb-cluster-monitoring-operator-tls podName:56f88c5a-6833-4eab-b858-8c4b163fbdcb nodeName:}" failed. No retries permitted until 2026-07-07 15:01:39.880213186 +0000 UTC m=+67.676749906 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/56f88c5a-6833-4eab-b858-8c4b163fbdcb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-qt79g" (UID: "56f88c5a-6833-4eab-b858-8c4b163fbdcb") : secret "cluster-monitoring-operator-tls" not found
Jul 07 15:01:23.880341 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:23.880251    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 15:01:23.880438 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:23.880429    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cfbcb4769-87kk4: secret "image-registry-tls" not found
Jul 07 15:01:23.880560 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:23.880552    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-tls podName:d050a3c0-f99d-4e6d-99f3-03ce677e7fa0 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:39.88053935 +0000 UTC m=+67.677076070 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-tls") pod "image-registry-6cfbcb4769-87kk4" (UID: "d050a3c0-f99d-4e6d-99f3-03ce677e7fa0") : secret "image-registry-tls" not found
Jul 07 15:01:23.880651 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:23.880306    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 15:01:23.880768 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:23.880760    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44e4a5b8-14bf-482c-9297-f2809ddb36c1-metrics-certs podName:44e4a5b8-14bf-482c-9297-f2809ddb36c1 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:39.880748701 +0000 UTC m=+67.677285423 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/44e4a5b8-14bf-482c-9297-f2809ddb36c1-metrics-certs") pod "router-default-58948c8cff-pclzr" (UID: "44e4a5b8-14bf-482c-9297-f2809ddb36c1") : secret "router-metrics-certs-default" not found
Jul 07 15:01:23.981177 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:23.981075    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-f4wfx\" (UID: \"9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f4wfx"
Jul 07 15:01:23.983757 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:23.983727    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-f4wfx\" (UID: \"9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f4wfx"
Jul 07 15:01:23.992525 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:23.992501    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f4wfx"
Jul 07 15:01:24.082810 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:24.081956    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23819f39-1440-4b6c-90fa-6db19eb48b74-metrics-tls\") pod \"dns-default-rlrhq\" (UID: \"23819f39-1440-4b6c-90fa-6db19eb48b74\") " pod="openshift-dns/dns-default-rlrhq"
Jul 07 15:01:24.082810 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:24.082246    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 15:01:24.082810 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:24.082302    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23819f39-1440-4b6c-90fa-6db19eb48b74-metrics-tls podName:23819f39-1440-4b6c-90fa-6db19eb48b74 nodeName:}" failed. No retries permitted until 2026-07-07 15:01:40.082284718 +0000 UTC m=+67.878821430 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/23819f39-1440-4b6c-90fa-6db19eb48b74-metrics-tls") pod "dns-default-rlrhq" (UID: "23819f39-1440-4b6c-90fa-6db19eb48b74") : secret "dns-default-metrics-tls" not found
Jul 07 15:01:24.193960 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:24.193884    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f4wfx"]
Jul 07 15:01:24.586792 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:24.586756    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c81a8d96-38ae-48b5-bd30-2711751dedef-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6m26d\" (UID: \"c81a8d96-38ae-48b5-bd30-2711751dedef\") " pod="openshift-insights/insights-runtime-extractor-6m26d"
Jul 07 15:01:24.586979 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:24.586802    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c81a8d96-38ae-48b5-bd30-2711751dedef-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6m26d\" (UID: \"c81a8d96-38ae-48b5-bd30-2711751dedef\") " pod="openshift-insights/insights-runtime-extractor-6m26d"
Jul 07 15:01:24.586979 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:24.586956    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 15:01:24.587057 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:24.587012    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c81a8d96-38ae-48b5-bd30-2711751dedef-insights-runtime-extractor-tls podName:c81a8d96-38ae-48b5-bd30-2711751dedef nodeName:}" failed. No retries permitted until 2026-07-07 15:01:28.586996965 +0000 UTC m=+56.383533674 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c81a8d96-38ae-48b5-bd30-2711751dedef-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6m26d" (UID: "c81a8d96-38ae-48b5-bd30-2711751dedef") : secret "insights-runtime-extractor-tls" not found
Jul 07 15:01:24.587351 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:24.587332    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c81a8d96-38ae-48b5-bd30-2711751dedef-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6m26d\" (UID: \"c81a8d96-38ae-48b5-bd30-2711751dedef\") " pod="openshift-insights/insights-runtime-extractor-6m26d"
Jul 07 15:01:25.093277 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:25.093238    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f4wfx" event={"ID":"9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118","Type":"ContainerStarted","Data":"2ebe9bb73e27f6ae96ba5a58e3393538b4690f4c9c72f8a34a70fadaf9d55e48"}
Jul 07 15:01:27.102167 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:27.102065    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f4wfx" event={"ID":"9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118","Type":"ContainerStarted","Data":"943a8dcf3abcb6d5733c065e1a5018ddaec8290eee000c2e59beb8aba2ae2114"}
Jul 07 15:01:27.102167 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:27.102140    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f4wfx" event={"ID":"9afbd5a6-cdc1-4ccc-b6a3-35b5a92bb118","Type":"ContainerStarted","Data":"e07f005eb9f91a3931081cc1a940e17a179f1b2f30e1e0b1bdb81127a4ccc678"}
Jul 07 15:01:27.128610 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:27.128552    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-f4wfx" podStartSLOduration=26.581261501 podStartE2EDuration="29.128530689s" totalImagesPullingTime="2.547269188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:00:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:27.12588731 +0000 UTC m=+54.922424058" watchObservedRunningTime="2026-07-07 15:01:27.128530689 +0000 UTC m=+54.925067418"
Jul 07 15:01:28.418920 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:28.418878    2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-5r9rp"
Jul 07 15:01:28.419297 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:28.419261    2567 scope.go:117] "RemoveContainer" containerID="1e4b1ea7047fd6028f958306e7ff1d8faae388a16124e236b9526c396317efb2"
Jul 07 15:01:28.419429 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:28.419411    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-5r9rp_openshift-console-operator(1c51cd93-b0ea-4aa1-8678-a4c833b2d37e)\"" pod="openshift-console-operator/console-operator-68b757fb67-5r9rp" podUID="1c51cd93-b0ea-4aa1-8678-a4c833b2d37e"
Jul 07 15:01:28.621212 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:28.621175    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c81a8d96-38ae-48b5-bd30-2711751dedef-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6m26d\" (UID: \"c81a8d96-38ae-48b5-bd30-2711751dedef\") " pod="openshift-insights/insights-runtime-extractor-6m26d"
Jul 07 15:01:28.621392 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:28.621317    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 15:01:28.621392 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:28.621379    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c81a8d96-38ae-48b5-bd30-2711751dedef-insights-runtime-extractor-tls podName:c81a8d96-38ae-48b5-bd30-2711751dedef nodeName:}" failed. No retries permitted until 2026-07-07 15:01:36.621364346 +0000 UTC m=+64.417901052 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c81a8d96-38ae-48b5-bd30-2711751dedef-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6m26d" (UID: "c81a8d96-38ae-48b5-bd30-2711751dedef") : secret "insights-runtime-extractor-tls" not found
Jul 07 15:01:29.048427 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:29.048385    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-5r9rp"
Jul 07 15:01:29.108613 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:29.108585    2567 scope.go:117] "RemoveContainer" containerID="1e4b1ea7047fd6028f958306e7ff1d8faae388a16124e236b9526c396317efb2"
Jul 07 15:01:29.108802 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:01:29.108777    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-5r9rp_openshift-console-operator(1c51cd93-b0ea-4aa1-8678-a4c833b2d37e)\"" pod="openshift-console-operator/console-operator-68b757fb67-5r9rp" podUID="1c51cd93-b0ea-4aa1-8678-a4c833b2d37e"
Jul 07 15:01:33.972617 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:33.972586    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xj52l"
Jul 07 15:01:36.690823 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:36.690777    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c81a8d96-38ae-48b5-bd30-2711751dedef-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6m26d\" (UID: \"c81a8d96-38ae-48b5-bd30-2711751dedef\") " pod="openshift-insights/insights-runtime-extractor-6m26d"
Jul 07 15:01:36.693183 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:36.693163    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c81a8d96-38ae-48b5-bd30-2711751dedef-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6m26d\" (UID: \"c81a8d96-38ae-48b5-bd30-2711751dedef\") " pod="openshift-insights/insights-runtime-extractor-6m26d"
Jul 07 15:01:36.785009 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:36.784974    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4zds6\""
Jul 07 15:01:36.792045 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:36.792012    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6m26d"
Jul 07 15:01:36.930338 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:36.930304    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6m26d"]
Jul 07 15:01:36.935155 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:01:36.935126    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc81a8d96_38ae_48b5_bd30_2711751dedef.slice/crio-0a0dbfcc125ae295f36a45d327d40e435a761815dc29e93dddef263a2960901f WatchSource:0}: Error finding container 0a0dbfcc125ae295f36a45d327d40e435a761815dc29e93dddef263a2960901f: Status 404 returned error can't find the container with id 0a0dbfcc125ae295f36a45d327d40e435a761815dc29e93dddef263a2960901f
Jul 07 15:01:37.137981 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:37.137944    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6m26d" event={"ID":"c81a8d96-38ae-48b5-bd30-2711751dedef","Type":"ContainerStarted","Data":"0a0dbfcc125ae295f36a45d327d40e435a761815dc29e93dddef263a2960901f"}
Jul 07 15:01:38.407041 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:38.406999    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-original-pull-secret\") pod \"global-pull-secret-syncer-zz5ql\" (UID: \"31b4f468-dc43-4504-b8c1-3f6b91c08bc5\") " pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:01:38.411389 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:38.411367    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 15:01:38.419798 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:38.419779    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/31b4f468-dc43-4504-b8c1-3f6b91c08bc5-original-pull-secret\") pod \"global-pull-secret-syncer-zz5ql\" (UID: \"31b4f468-dc43-4504-b8c1-3f6b91c08bc5\") " pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:01:38.508158 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:38.508123    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ebbf4a6-429c-465d-916d-84567a210db5-metrics-certs\") pod \"network-metrics-daemon-zs477\" (UID: \"0ebbf4a6-429c-465d-916d-84567a210db5\") " pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:01:38.512150 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:38.512125    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 15:01:38.521601 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:38.521580    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ebbf4a6-429c-465d-916d-84567a210db5-metrics-certs\") pod \"network-metrics-daemon-zs477\" (UID: \"0ebbf4a6-429c-465d-916d-84567a210db5\") " pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:01:38.586974 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:38.586934    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zz5ql"
Jul 07 15:01:38.609571 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:38.609535    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvn2q\" (UniqueName: \"kubernetes.io/projected/a7a85a73-907b-4a3b-bee4-b03ce0da2b6c-kube-api-access-jvn2q\") pod \"network-check-target-9gmwd\" (UID: \"a7a85a73-907b-4a3b-bee4-b03ce0da2b6c\") " pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:01:38.612714 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:38.612688    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvn2q\" (UniqueName: \"kubernetes.io/projected/a7a85a73-907b-4a3b-bee4-b03ce0da2b6c-kube-api-access-jvn2q\") pod \"network-check-target-9gmwd\" (UID: \"a7a85a73-907b-4a3b-bee4-b03ce0da2b6c\") " pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:01:38.748560 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:38.748528    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zz5ql"]
Jul 07 15:01:38.752321 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:01:38.752296    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31b4f468_dc43_4504_b8c1_3f6b91c08bc5.slice/crio-565e5e2599a9724f5f12a6b2c33b7f8b0485bac4d8fd1834d1c5f7ac05560927 WatchSource:0}: Error finding container 565e5e2599a9724f5f12a6b2c33b7f8b0485bac4d8fd1834d1c5f7ac05560927: Status 404 returned error can't find the container with id 565e5e2599a9724f5f12a6b2c33b7f8b0485bac4d8fd1834d1c5f7ac05560927
Jul 07 15:01:38.788173 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:38.788149    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ghr77\""
Jul 07 15:01:38.795057 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:38.795033    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zs477"
Jul 07 15:01:38.795194 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:38.795100    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bchjv\""
Jul 07 15:01:38.803073 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:38.803044    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:01:38.969361 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:38.969330    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zs477"]
Jul 07 15:01:38.972029 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:01:38.972000    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ebbf4a6_429c_465d_916d_84567a210db5.slice/crio-d2e9ee8968f4c3fc2d058c460f7f88f91065b69716c0372aa5001e5419ab4e05 WatchSource:0}: Error finding container d2e9ee8968f4c3fc2d058c460f7f88f91065b69716c0372aa5001e5419ab4e05: Status 404 returned error can't find the container with id d2e9ee8968f4c3fc2d058c460f7f88f91065b69716c0372aa5001e5419ab4e05
Jul 07 15:01:38.987185 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:38.987153    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9gmwd"]
Jul 07 15:01:38.990582 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:01:38.990554    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7a85a73_907b_4a3b_bee4_b03ce0da2b6c.slice/crio-642eb945caddfb28beefaa45d2dad9242e7f08b5d65fbb31246e819e01cc788d WatchSource:0}: Error finding container 642eb945caddfb28beefaa45d2dad9242e7f08b5d65fbb31246e819e01cc788d: Status 404 returned error can't find the container with id 642eb945caddfb28beefaa45d2dad9242e7f08b5d65fbb31246e819e01cc788d
Jul 07 15:01:39.146194 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:39.146159    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9gmwd" event={"ID":"a7a85a73-907b-4a3b-bee4-b03ce0da2b6c","Type":"ContainerStarted","Data":"3024134bcbb07287477a00501fa16b4eebecce68ed2a2ad52cfce5be41de0c45"}
Jul 07 15:01:39.146401 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:39.146317    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:01:39.146401 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:39.146352    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9gmwd" event={"ID":"a7a85a73-907b-4a3b-bee4-b03ce0da2b6c","Type":"ContainerStarted","Data":"642eb945caddfb28beefaa45d2dad9242e7f08b5d65fbb31246e819e01cc788d"}
Jul 07 15:01:39.147405 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:39.147384    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zs477" event={"ID":"0ebbf4a6-429c-465d-916d-84567a210db5","Type":"ContainerStarted","Data":"d2e9ee8968f4c3fc2d058c460f7f88f91065b69716c0372aa5001e5419ab4e05"}
Jul 07 15:01:39.148448 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:39.148431    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zz5ql" event={"ID":"31b4f468-dc43-4504-b8c1-3f6b91c08bc5","Type":"ContainerStarted","Data":"565e5e2599a9724f5f12a6b2c33b7f8b0485bac4d8fd1834d1c5f7ac05560927"}
Jul 07 15:01:39.173919 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:39.173871    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9gmwd" podStartSLOduration=66.173846889 podStartE2EDuration="1m6.173846889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:00:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:39.1732666 +0000 UTC m=+66.969803344" watchObservedRunningTime="2026-07-07 15:01:39.173846889 +0000 UTC m=+66.970383607"
Jul 07 15:01:39.920695 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:39.920650    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44e4a5b8-14bf-482c-9297-f2809ddb36c1-metrics-certs\") pod \"router-default-58948c8cff-pclzr\" (UID: \"44e4a5b8-14bf-482c-9297-f2809ddb36c1\") " pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:39.921189 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:39.920748    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/56f88c5a-6833-4eab-b858-8c4b163fbdcb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-qt79g\" (UID: \"56f88c5a-6833-4eab-b858-8c4b163fbdcb\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qt79g"
Jul 07 15:01:39.921189 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:39.920782    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-tls\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:39.921189 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:39.920807    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44e4a5b8-14bf-482c-9297-f2809ddb36c1-service-ca-bundle\") pod \"router-default-58948c8cff-pclzr\" (UID: \"44e4a5b8-14bf-482c-9297-f2809ddb36c1\") " pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:39.921863 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:39.921672    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44e4a5b8-14bf-482c-9297-f2809ddb36c1-service-ca-bundle\") pod \"router-default-58948c8cff-pclzr\" (UID: \"44e4a5b8-14bf-482c-9297-f2809ddb36c1\") " pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:39.923638 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:39.923594    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/44e4a5b8-14bf-482c-9297-f2809ddb36c1-metrics-certs\") pod \"router-default-58948c8cff-pclzr\" (UID: \"44e4a5b8-14bf-482c-9297-f2809ddb36c1\") " pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:39.923763 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:39.923744    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/56f88c5a-6833-4eab-b858-8c4b163fbdcb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-qt79g\" (UID: \"56f88c5a-6833-4eab-b858-8c4b163fbdcb\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qt79g"
Jul 07 15:01:39.924228 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:39.924210    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-tls\") pod \"image-registry-6cfbcb4769-87kk4\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") " pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:40.123343 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:40.123309    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23819f39-1440-4b6c-90fa-6db19eb48b74-metrics-tls\") pod \"dns-default-rlrhq\" (UID: \"23819f39-1440-4b6c-90fa-6db19eb48b74\") " pod="openshift-dns/dns-default-rlrhq"
Jul 07 15:01:40.126247 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:40.126221    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23819f39-1440-4b6c-90fa-6db19eb48b74-metrics-tls\") pod \"dns-default-rlrhq\" (UID: \"23819f39-1440-4b6c-90fa-6db19eb48b74\") " pod="openshift-dns/dns-default-rlrhq"
Jul 07 15:01:40.152134 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:40.152101    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xfh92\""
Jul 07 15:01:40.161263 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:40.159357    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qt79g"
Jul 07 15:01:40.162247 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:40.162218    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-ts8vc\""
Jul 07 15:01:40.168753 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:40.168716    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:40.171955 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:40.171898    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-t8hm9\""
Jul 07 15:01:40.179695 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:40.179659    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:40.271528 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:40.271498    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6qs2h\""
Jul 07 15:01:40.279468 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:40.279424    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rlrhq"
Jul 07 15:01:41.793788 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.793621    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-pfl6c"]
Jul 07 15:01:41.796046 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.796025    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7497d66c94-q942f"]
Jul 07 15:01:41.796157 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.796138    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-pfl6c"
Jul 07 15:01:41.798485 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.798466    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7497d66c94-q942f"
Jul 07 15:01:41.799426 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.799402    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f57ff8d78-dnmf6"]
Jul 07 15:01:41.802770 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.802520    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f57ff8d78-dnmf6"
Jul 07 15:01:41.804520 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.804453    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 07 15:01:41.804665 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.804603    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 07 15:01:41.804736 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.804726    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 07 15:01:41.805162 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.804824    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 07 15:01:41.805162 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.804945    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-w6c8p\""
Jul 07 15:01:41.805162 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.805061    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 07 15:01:41.805162 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.805142    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-xw4gh\""
Jul 07 15:01:41.806008 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.805746    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 07 15:01:41.806008 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.805775    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 07 15:01:41.806350 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.806330    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc66794f-l88nv"]
Jul 07 15:01:41.810281 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.810262    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc66794f-l88nv"
Jul 07 15:01:41.832862 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.831797    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-pfl6c"]
Jul 07 15:01:41.841869 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.836087    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 07 15:01:41.841869 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.836483    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 07 15:01:41.841869 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.839976    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 07 15:01:41.841869 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.840186    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 07 15:01:41.844701 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.844663    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f57ff8d78-dnmf6"]
Jul 07 15:01:41.844701 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.844706    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7497d66c94-q942f"]
Jul 07 15:01:41.850292 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.850270    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc66794f-l88nv"]
Jul 07 15:01:41.908491 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.908455    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-qzhfn"]
Jul 07 15:01:41.912197 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.912168    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-qzhfn"
Jul 07 15:01:41.916924 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.916899    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 07 15:01:41.924013 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.923988    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 07 15:01:41.924138 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.924079    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-x2hft\""
Jul 07 15:01:41.941927 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.941901    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/636cab31-5bac-436d-92e7-49a2e798e984-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-pfl6c\" (UID: \"636cab31-5bac-436d-92e7-49a2e798e984\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-pfl6c"
Jul 07 15:01:41.942069 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.941940    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a2edf693-41a7-449b-80d7-fb05e08e0e78-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7fdc66794f-l88nv\" (UID: \"a2edf693-41a7-449b-80d7-fb05e08e0e78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc66794f-l88nv"
Jul 07 15:01:41.942069 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.941972    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qp27c\" (UniqueName: \"kubernetes.io/projected/ecc8b759-1302-478f-a51b-aa782672b818-kube-api-access-qp27c\") pod \"managed-serviceaccount-addon-agent-7497d66c94-q942f\" (UID: \"ecc8b759-1302-478f-a51b-aa782672b818\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7497d66c94-q942f"
Jul 07 15:01:41.942069 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.941998    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a2edf693-41a7-449b-80d7-fb05e08e0e78-hub\") pod \"cluster-proxy-proxy-agent-7fdc66794f-l88nv\" (UID: \"a2edf693-41a7-449b-80d7-fb05e08e0e78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc66794f-l88nv"
Jul 07 15:01:41.942069 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.942021    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rmhx\" (UniqueName: \"kubernetes.io/projected/a2edf693-41a7-449b-80d7-fb05e08e0e78-kube-api-access-9rmhx\") pod \"cluster-proxy-proxy-agent-7fdc66794f-l88nv\" (UID: \"a2edf693-41a7-449b-80d7-fb05e08e0e78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc66794f-l88nv"
Jul 07 15:01:41.942069 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.942049    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/636cab31-5bac-436d-92e7-49a2e798e984-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-pfl6c\" (UID: \"636cab31-5bac-436d-92e7-49a2e798e984\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-pfl6c"
Jul 07 15:01:41.942305 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.942086    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86273f20-948b-4ff4-a02f-007646e091d9-tmp\") pod \"klusterlet-addon-workmgr-7f57ff8d78-dnmf6\" (UID: \"86273f20-948b-4ff4-a02f-007646e091d9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f57ff8d78-dnmf6"
Jul 07 15:01:41.942305 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.942180    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a2edf693-41a7-449b-80d7-fb05e08e0e78-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7fdc66794f-l88nv\" (UID: \"a2edf693-41a7-449b-80d7-fb05e08e0e78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc66794f-l88nv"
Jul 07 15:01:41.942305 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.942224    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2edf693-41a7-449b-80d7-fb05e08e0e78-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7fdc66794f-l88nv\" (UID: \"a2edf693-41a7-449b-80d7-fb05e08e0e78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc66794f-l88nv"
Jul 07 15:01:41.942305 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.942277    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a2edf693-41a7-449b-80d7-fb05e08e0e78-ca\") pod \"cluster-proxy-proxy-agent-7fdc66794f-l88nv\" (UID: \"a2edf693-41a7-449b-80d7-fb05e08e0e78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc66794f-l88nv"
Jul 07 15:01:41.942467 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.942308    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjpmb\" (UniqueName: \"kubernetes.io/projected/86273f20-948b-4ff4-a02f-007646e091d9-kube-api-access-jjpmb\") pod \"klusterlet-addon-workmgr-7f57ff8d78-dnmf6\" (UID: \"86273f20-948b-4ff4-a02f-007646e091d9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f57ff8d78-dnmf6"
Jul 07 15:01:41.942467 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.942335    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ecc8b759-1302-478f-a51b-aa782672b818-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7497d66c94-q942f\" (UID: \"ecc8b759-1302-478f-a51b-aa782672b818\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7497d66c94-q942f"
Jul 07 15:01:41.942467 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:41.942364    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/86273f20-948b-4ff4-a02f-007646e091d9-klusterlet-config\") pod \"klusterlet-addon-workmgr-7f57ff8d78-dnmf6\" (UID: \"86273f20-948b-4ff4-a02f-007646e091d9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f57ff8d78-dnmf6"
Jul 07 15:01:42.001549 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.001514    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-qzhfn"]
Jul 07 15:01:42.042873 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.042820    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a2edf693-41a7-449b-80d7-fb05e08e0e78-ca\") pod \"cluster-proxy-proxy-agent-7fdc66794f-l88nv\" (UID: \"a2edf693-41a7-449b-80d7-fb05e08e0e78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc66794f-l88nv"
Jul 07 15:01:42.043064 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.042896    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjpmb\" (UniqueName: \"kubernetes.io/projected/86273f20-948b-4ff4-a02f-007646e091d9-kube-api-access-jjpmb\") pod \"klusterlet-addon-workmgr-7f57ff8d78-dnmf6\" (UID: \"86273f20-948b-4ff4-a02f-007646e091d9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f57ff8d78-dnmf6"
Jul 07 15:01:42.043064 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.043049    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ecc8b759-1302-478f-a51b-aa782672b818-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7497d66c94-q942f\" (UID: \"ecc8b759-1302-478f-a51b-aa782672b818\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7497d66c94-q942f"
Jul 07 15:01:42.043181 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.043100    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/86273f20-948b-4ff4-a02f-007646e091d9-klusterlet-config\") pod \"klusterlet-addon-workmgr-7f57ff8d78-dnmf6\" (UID: \"86273f20-948b-4ff4-a02f-007646e091d9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f57ff8d78-dnmf6"
Jul 07 15:01:42.043181 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.043136    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/636cab31-5bac-436d-92e7-49a2e798e984-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-pfl6c\" (UID: \"636cab31-5bac-436d-92e7-49a2e798e984\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-pfl6c"
Jul 07 15:01:42.043181 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.043157    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a2edf693-41a7-449b-80d7-fb05e08e0e78-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7fdc66794f-l88nv\" (UID: \"a2edf693-41a7-449b-80d7-fb05e08e0e78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc66794f-l88nv"
Jul 07 15:01:42.043340 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.043192    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qp27c\" (UniqueName: \"kubernetes.io/projected/ecc8b759-1302-478f-a51b-aa782672b818-kube-api-access-qp27c\") pod \"managed-serviceaccount-addon-agent-7497d66c94-q942f\" (UID: \"ecc8b759-1302-478f-a51b-aa782672b818\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7497d66c94-q942f"
Jul 07 15:01:42.043340 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.043220    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a2edf693-41a7-449b-80d7-fb05e08e0e78-hub\") pod \"cluster-proxy-proxy-agent-7fdc66794f-l88nv\" (UID: \"a2edf693-41a7-449b-80d7-fb05e08e0e78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc66794f-l88nv"
Jul 07 15:01:42.043340 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.043254    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rmhx\" (UniqueName: \"kubernetes.io/projected/a2edf693-41a7-449b-80d7-fb05e08e0e78-kube-api-access-9rmhx\") pod \"cluster-proxy-proxy-agent-7fdc66794f-l88nv\" (UID: \"a2edf693-41a7-449b-80d7-fb05e08e0e78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc66794f-l88nv"
Jul 07 15:01:42.043340 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.043294    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/636cab31-5bac-436d-92e7-49a2e798e984-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-pfl6c\" (UID: \"636cab31-5bac-436d-92e7-49a2e798e984\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-pfl6c"
Jul 07 15:01:42.043340 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.043324    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2rfv\" (UniqueName: \"kubernetes.io/projected/e09c67d4-41b0-46a7-81a4-b556e763a739-kube-api-access-h2rfv\") pod \"migrator-d57d58cc9-qzhfn\" (UID: \"e09c67d4-41b0-46a7-81a4-b556e763a739\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-qzhfn"
Jul 07 15:01:42.043643 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.043362    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86273f20-948b-4ff4-a02f-007646e091d9-tmp\") pod \"klusterlet-addon-workmgr-7f57ff8d78-dnmf6\" (UID: \"86273f20-948b-4ff4-a02f-007646e091d9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f57ff8d78-dnmf6"
Jul 07 15:01:42.043643 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.043402    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a2edf693-41a7-449b-80d7-fb05e08e0e78-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7fdc66794f-l88nv\" (UID: \"a2edf693-41a7-449b-80d7-fb05e08e0e78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc66794f-l88nv"
Jul 07 15:01:42.043643 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.043439    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2edf693-41a7-449b-80d7-fb05e08e0e78-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7fdc66794f-l88nv\" (UID: \"a2edf693-41a7-449b-80d7-fb05e08e0e78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc66794f-l88nv"
Jul 07 15:01:42.044563 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.044235    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2edf693-41a7-449b-80d7-fb05e08e0e78-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7fdc66794f-l88nv\" (UID: \"a2edf693-41a7-449b-80d7-fb05e08e0e78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc66794f-l88nv"
Jul 07 15:01:42.044563 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.044436    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/636cab31-5bac-436d-92e7-49a2e798e984-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-pfl6c\" (UID: \"636cab31-5bac-436d-92e7-49a2e798e984\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-pfl6c"
Jul 07 15:01:42.044563 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.044478    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86273f20-948b-4ff4-a02f-007646e091d9-tmp\") pod \"klusterlet-addon-workmgr-7f57ff8d78-dnmf6\" (UID: \"86273f20-948b-4ff4-a02f-007646e091d9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f57ff8d78-dnmf6"
Jul 07 15:01:42.046776 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.046556    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/636cab31-5bac-436d-92e7-49a2e798e984-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-pfl6c\" (UID: \"636cab31-5bac-436d-92e7-49a2e798e984\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-pfl6c"
Jul 07 15:01:42.046776 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.046575    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ecc8b759-1302-478f-a51b-aa782672b818-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7497d66c94-q942f\" (UID: \"ecc8b759-1302-478f-a51b-aa782672b818\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7497d66c94-q942f"
Jul 07 15:01:42.047432 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.047405    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/86273f20-948b-4ff4-a02f-007646e091d9-klusterlet-config\") pod \"klusterlet-addon-workmgr-7f57ff8d78-dnmf6\" (UID: \"86273f20-948b-4ff4-a02f-007646e091d9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f57ff8d78-dnmf6"
Jul 07 15:01:42.047937 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.047910    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a2edf693-41a7-449b-80d7-fb05e08e0e78-ca\") pod \"cluster-proxy-proxy-agent-7fdc66794f-l88nv\" (UID: \"a2edf693-41a7-449b-80d7-fb05e08e0e78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc66794f-l88nv"
Jul 07 15:01:42.048035 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.047925    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a2edf693-41a7-449b-80d7-fb05e08e0e78-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7fdc66794f-l88nv\" (UID: \"a2edf693-41a7-449b-80d7-fb05e08e0e78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc66794f-l88nv"
Jul 07 15:01:42.048035 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.047963    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a2edf693-41a7-449b-80d7-fb05e08e0e78-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7fdc66794f-l88nv\" (UID: \"a2edf693-41a7-449b-80d7-fb05e08e0e78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc66794f-l88nv"
Jul 07 15:01:42.048035 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.047977    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a2edf693-41a7-449b-80d7-fb05e08e0e78-hub\") pod \"cluster-proxy-proxy-agent-7fdc66794f-l88nv\" (UID: \"a2edf693-41a7-449b-80d7-fb05e08e0e78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc66794f-l88nv"
Jul 07 15:01:42.058992 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.058937    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjpmb\" (UniqueName: \"kubernetes.io/projected/86273f20-948b-4ff4-a02f-007646e091d9-kube-api-access-jjpmb\") pod \"klusterlet-addon-workmgr-7f57ff8d78-dnmf6\" (UID: \"86273f20-948b-4ff4-a02f-007646e091d9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f57ff8d78-dnmf6"
Jul 07 15:01:42.071918 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.071887    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rmhx\" (UniqueName: \"kubernetes.io/projected/a2edf693-41a7-449b-80d7-fb05e08e0e78-kube-api-access-9rmhx\") pod \"cluster-proxy-proxy-agent-7fdc66794f-l88nv\" (UID: \"a2edf693-41a7-449b-80d7-fb05e08e0e78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc66794f-l88nv"
Jul 07 15:01:42.076337 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.076284    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qp27c\" (UniqueName: \"kubernetes.io/projected/ecc8b759-1302-478f-a51b-aa782672b818-kube-api-access-qp27c\") pod \"managed-serviceaccount-addon-agent-7497d66c94-q942f\" (UID: \"ecc8b759-1302-478f-a51b-aa782672b818\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7497d66c94-q942f"
Jul 07 15:01:42.108013 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.107979    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-pfl6c"
Jul 07 15:01:42.129645 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.129617    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7497d66c94-q942f"
Jul 07 15:01:42.143971 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.143940    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2rfv\" (UniqueName: \"kubernetes.io/projected/e09c67d4-41b0-46a7-81a4-b556e763a739-kube-api-access-h2rfv\") pod \"migrator-d57d58cc9-qzhfn\" (UID: \"e09c67d4-41b0-46a7-81a4-b556e763a739\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-qzhfn"
Jul 07 15:01:42.154509 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.154487    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f57ff8d78-dnmf6"
Jul 07 15:01:42.159678 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.159657    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc66794f-l88nv"
Jul 07 15:01:42.160100 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.160069    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2rfv\" (UniqueName: \"kubernetes.io/projected/e09c67d4-41b0-46a7-81a4-b556e763a739-kube-api-access-h2rfv\") pod \"migrator-d57d58cc9-qzhfn\" (UID: \"e09c67d4-41b0-46a7-81a4-b556e763a739\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-qzhfn"
Jul 07 15:01:42.238504 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:42.238471    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-qzhfn"
Jul 07 15:01:43.423152 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:43.423122    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qt79g"]
Jul 07 15:01:43.547605 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:01:43.547560    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56f88c5a_6833_4eab_b858_8c4b163fbdcb.slice/crio-34d8a602bd33e404c80b45e17a800286f4ec64b81c07a099143e695ab0e93f7f WatchSource:0}: Error finding container 34d8a602bd33e404c80b45e17a800286f4ec64b81c07a099143e695ab0e93f7f: Status 404 returned error can't find the container with id 34d8a602bd33e404c80b45e17a800286f4ec64b81c07a099143e695ab0e93f7f
Jul 07 15:01:43.763466 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:43.763375    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cfbcb4769-87kk4"]
Jul 07 15:01:43.774112 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:43.772556    2567 scope.go:117] "RemoveContainer" containerID="1e4b1ea7047fd6028f958306e7ff1d8faae388a16124e236b9526c396317efb2"
Jul 07 15:01:43.836011 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:43.835908    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rlrhq"]
Jul 07 15:01:43.841944 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:43.841353    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-58948c8cff-pclzr"]
Jul 07 15:01:43.885302 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:43.885236    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc66794f-l88nv"]
Jul 07 15:01:43.900437 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:01:43.899180    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2edf693_41a7_449b_80d7_fb05e08e0e78.slice/crio-8732ced820f6a68e418b9e8d97aa236ae5fa5ea0e02dd2373fc94f2a63b2620b WatchSource:0}: Error finding container 8732ced820f6a68e418b9e8d97aa236ae5fa5ea0e02dd2373fc94f2a63b2620b: Status 404 returned error can't find the container with id 8732ced820f6a68e418b9e8d97aa236ae5fa5ea0e02dd2373fc94f2a63b2620b
Jul 07 15:01:43.931044 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:43.931023    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-pfl6c"]
Jul 07 15:01:43.936401 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:01:43.936367    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod636cab31_5bac_436d_92e7_49a2e798e984.slice/crio-048c90b6009f950ebb25245ffa5677005c2cde61583dced94b7f76fbdcebab2b WatchSource:0}: Error finding container 048c90b6009f950ebb25245ffa5677005c2cde61583dced94b7f76fbdcebab2b: Status 404 returned error can't find the container with id 048c90b6009f950ebb25245ffa5677005c2cde61583dced94b7f76fbdcebab2b
Jul 07 15:01:44.134808 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:44.133616    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-qzhfn"]
Jul 07 15:01:44.136950 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:44.135261    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7497d66c94-q942f"]
Jul 07 15:01:44.140043 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:44.139928    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f57ff8d78-dnmf6"]
Jul 07 15:01:44.149750 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:01:44.149686    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86273f20_948b_4ff4_a02f_007646e091d9.slice/crio-1a2b30c3f9e555b14073de65145df1c8142987dec62fcabecc1f007e1ce65f00 WatchSource:0}: Error finding container 1a2b30c3f9e555b14073de65145df1c8142987dec62fcabecc1f007e1ce65f00: Status 404 returned error can't find the container with id 1a2b30c3f9e555b14073de65145df1c8142987dec62fcabecc1f007e1ce65f00
Jul 07 15:01:44.176429 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:44.176404    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5r9rp_1c51cd93-b0ea-4aa1-8678-a4c833b2d37e/console-operator/1.log"
Jul 07 15:01:44.176566 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:44.176475    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-5r9rp" event={"ID":"1c51cd93-b0ea-4aa1-8678-a4c833b2d37e","Type":"ContainerStarted","Data":"ea36fc8b98bd4a7c4ae3c09f7e977438ebda0b9df4cfee88027d87ddbb8c4f2b"}
Jul 07 15:01:44.177217 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:44.176951    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-5r9rp"
Jul 07 15:01:44.177857 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:44.177815    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-qzhfn" event={"ID":"e09c67d4-41b0-46a7-81a4-b556e763a739","Type":"ContainerStarted","Data":"14d3c49d223357891ba982b31a3ef91736f185eeb7c1d4d7777ddbf38a5af414"}
Jul 07 15:01:44.179223 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:44.179195    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f57ff8d78-dnmf6" event={"ID":"86273f20-948b-4ff4-a02f-007646e091d9","Type":"ContainerStarted","Data":"1a2b30c3f9e555b14073de65145df1c8142987dec62fcabecc1f007e1ce65f00"}
Jul 07 15:01:44.180386 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:44.180365    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qt79g" event={"ID":"56f88c5a-6833-4eab-b858-8c4b163fbdcb","Type":"ContainerStarted","Data":"34d8a602bd33e404c80b45e17a800286f4ec64b81c07a099143e695ab0e93f7f"}
Jul 07 15:01:44.186905 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:44.186868    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zs477" event={"ID":"0ebbf4a6-429c-465d-916d-84567a210db5","Type":"ContainerStarted","Data":"f12e672fd43a2f6073c9f31eeaa89866fa552bd8483f638f120adf9feefe2b5d"}
Jul 07 15:01:44.187014 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:44.186933    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zs477" event={"ID":"0ebbf4a6-429c-465d-916d-84567a210db5","Type":"ContainerStarted","Data":"f5f3218d64617a7d14a9868a1d43408f04c9840207fcb71a79f12cd5c6eded51"}
Jul 07 15:01:44.188200 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:44.188175    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zz5ql" event={"ID":"31b4f468-dc43-4504-b8c1-3f6b91c08bc5","Type":"ContainerStarted","Data":"da5ec635c299859dc4bb3510206f34a4db87cc8703ef8d1d806f7882bba53a1f"}
Jul 07 15:01:44.189734 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:44.189712    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-58948c8cff-pclzr" event={"ID":"44e4a5b8-14bf-482c-9297-f2809ddb36c1","Type":"ContainerStarted","Data":"1c83f87c2f1fa0c3cc143006ceaa98e88a10ed53a0478f7134082be3655148e7"}
Jul 07 15:01:44.189876 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:44.189751    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-58948c8cff-pclzr" event={"ID":"44e4a5b8-14bf-482c-9297-f2809ddb36c1","Type":"ContainerStarted","Data":"bdc7c875e94609183c8d5d5f4fa2a467af16c612f7e595d16925e3d2b69158ef"}
Jul 07 15:01:44.191392 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:44.191370    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4" event={"ID":"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0","Type":"ContainerStarted","Data":"6c56dac1deb9c0aff7f0f1e9aa636af608c719046d571d9e70860e7111f85f37"}
Jul 07 15:01:44.191493 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:44.191412    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4" event={"ID":"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0","Type":"ContainerStarted","Data":"6b96dfa547dadd642ca42247b50e5662003875fd6ade50d5d90ea16b9a2fa700"}
Jul 07 15:01:44.191493 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:44.191462    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:01:44.192870 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:44.192850    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6m26d" event={"ID":"c81a8d96-38ae-48b5-bd30-2711751dedef","Type":"ContainerStarted","Data":"f1612817b817e350770a470f58fe4918d303c9d41a8ac22f87238cd60cad80fb"}
Jul 07 15:01:44.193913 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:44.193893    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rlrhq" event={"ID":"23819f39-1440-4b6c-90fa-6db19eb48b74","Type":"ContainerStarted","Data":"ba962fc2c02f5503a20d80f215171b53fb1d671c1614109ab187276c9e4cbb8b"}
Jul 07 15:01:44.195057 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:44.195030    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-pfl6c" event={"ID":"636cab31-5bac-436d-92e7-49a2e798e984","Type":"ContainerStarted","Data":"048c90b6009f950ebb25245ffa5677005c2cde61583dced94b7f76fbdcebab2b"}
Jul 07 15:01:44.196216 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:44.196195    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc66794f-l88nv" event={"ID":"a2edf693-41a7-449b-80d7-fb05e08e0e78","Type":"ContainerStarted","Data":"8732ced820f6a68e418b9e8d97aa236ae5fa5ea0e02dd2373fc94f2a63b2620b"}
Jul 07 15:01:44.197364 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:44.197341    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7497d66c94-q942f" event={"ID":"ecc8b759-1302-478f-a51b-aa782672b818","Type":"ContainerStarted","Data":"b93236d7a9a94e198c506a8fc9816e0014b6a3b837bb4272c1d02a7d139a8954"}
Jul 07 15:01:44.249971 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:44.249901    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4" podStartSLOduration=70.249812779 podStartE2EDuration="1m10.249812779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:00:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:44.227623129 +0000 UTC m=+72.024159858" watchObservedRunningTime="2026-07-07 15:01:44.249812779 +0000 UTC m=+72.046349510"
Jul 07 15:01:44.250542 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:44.250507    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zs477" podStartSLOduration=66.628032856 podStartE2EDuration="1m11.250494315s" totalImagesPullingTime="4.622461459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:00:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:44.248514175 +0000 UTC m=+72.045050923" watchObservedRunningTime="2026-07-07 15:01:44.250494315 +0000 UTC m=+72.047031046"
Jul 07 15:01:44.279231 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:44.279164    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-58948c8cff-pclzr" podStartSLOduration=49.279140852 podStartE2EDuration="49.279140852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:00:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:44.277311474 +0000 UTC m=+72.073848231" watchObservedRunningTime="2026-07-07 15:01:44.279140852 +0000 UTC m=+72.075677582"
Jul 07 15:01:44.299406 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:44.299345    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zz5ql" podStartSLOduration=66.470502806 podStartE2EDuration="1m11.29932248s" totalImagesPullingTime="4.828819674s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:00:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:44.298602345 +0000 UTC m=+72.095139094" watchObservedRunningTime="2026-07-07 15:01:44.29932248 +0000 UTC m=+72.095859209"
Jul 07 15:01:44.919978 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:44.919941    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-5r9rp"
Jul 07 15:01:45.150103 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:45.149751    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-chvmp"]
Jul 07 15:01:45.158340 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:45.157571    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-chvmp"
Jul 07 15:01:45.171593 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:45.169671    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:45.171593 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:45.172179    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:45.171593 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:45.172376    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-zsg86\""
Jul 07 15:01:45.171593 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:45.172629    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 07 15:01:45.171593 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:45.172381    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 07 15:01:45.181028 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:45.180490    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-chvmp"]
Jul 07 15:01:45.213935 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:45.212937    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:45.214432 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:45.214407    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-58948c8cff-pclzr"
Jul 07 15:01:45.282417 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:45.282215    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvzzh\" (UniqueName: \"kubernetes.io/projected/acef1cce-5704-4d95-9418-8e87b81831bb-kube-api-access-pvzzh\") pod \"downloads-685bfb5f64-chvmp\" (UID: \"acef1cce-5704-4d95-9418-8e87b81831bb\") " pod="openshift-console/downloads-685bfb5f64-chvmp"
Jul 07 15:01:45.383405 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:45.383353    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvzzh\" (UniqueName: \"kubernetes.io/projected/acef1cce-5704-4d95-9418-8e87b81831bb-kube-api-access-pvzzh\") pod \"downloads-685bfb5f64-chvmp\" (UID: \"acef1cce-5704-4d95-9418-8e87b81831bb\") " pod="openshift-console/downloads-685bfb5f64-chvmp"
Jul 07 15:01:45.421066 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:45.421000    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvzzh\" (UniqueName: \"kubernetes.io/projected/acef1cce-5704-4d95-9418-8e87b81831bb-kube-api-access-pvzzh\") pod \"downloads-685bfb5f64-chvmp\" (UID: \"acef1cce-5704-4d95-9418-8e87b81831bb\") " pod="openshift-console/downloads-685bfb5f64-chvmp"
Jul 07 15:01:45.477572 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:45.477205    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-chvmp"
Jul 07 15:01:45.872926 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:45.872823    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-chvmp"]
Jul 07 15:01:47.383994 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:01:47.383950    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacef1cce_5704_4d95_9418_8e87b81831bb.slice/crio-0b7a7a1378d820e31d5479bceb6d0f161788393419bb08bbad49236bc33e7aa5 WatchSource:0}: Error finding container 0b7a7a1378d820e31d5479bceb6d0f161788393419bb08bbad49236bc33e7aa5: Status 404 returned error can't find the container with id 0b7a7a1378d820e31d5479bceb6d0f161788393419bb08bbad49236bc33e7aa5
Jul 07 15:01:48.229332 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:48.229274    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-chvmp" event={"ID":"acef1cce-5704-4d95-9418-8e87b81831bb","Type":"ContainerStarted","Data":"0b7a7a1378d820e31d5479bceb6d0f161788393419bb08bbad49236bc33e7aa5"}
Jul 07 15:01:54.263815 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:54.263683    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-pfl6c" event={"ID":"636cab31-5bac-436d-92e7-49a2e798e984","Type":"ContainerStarted","Data":"e67c1ffb4e6a66be680b7c83ed27748c7b96c3f4dff07024061b39f095fc0d98"}
Jul 07 15:01:54.267614 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:54.267547    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc66794f-l88nv" event={"ID":"a2edf693-41a7-449b-80d7-fb05e08e0e78","Type":"ContainerStarted","Data":"0244973658d5ccac7498ce90fc52f58077e5f97dceabde6f6425f6336bd56a1b"}
Jul 07 15:01:54.267614 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:54.267587    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc66794f-l88nv" event={"ID":"a2edf693-41a7-449b-80d7-fb05e08e0e78","Type":"ContainerStarted","Data":"9d550b37e4ca2cc2285ef9bab28cbf38b14f22f3836419fe96db35c1d358b63c"}
Jul 07 15:01:54.273413 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:54.273376    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7497d66c94-q942f" event={"ID":"ecc8b759-1302-478f-a51b-aa782672b818","Type":"ContainerStarted","Data":"543e7e128371f3e85633f79b7ee335f1d3e35e61cb499dfe48c3c347e5d7df55"}
Jul 07 15:01:54.276095 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:54.276037    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-qzhfn" event={"ID":"e09c67d4-41b0-46a7-81a4-b556e763a739","Type":"ContainerStarted","Data":"6c1938171a94f5a5855794a683d2ea31818b6e2453f990b18448e5af6605db89"}
Jul 07 15:01:54.276095 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:54.276089    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-qzhfn" event={"ID":"e09c67d4-41b0-46a7-81a4-b556e763a739","Type":"ContainerStarted","Data":"0e27079773bf0ec9c6bb60d3f49a893a73f4b89a6bfd5fa1e06af52e6508e92a"}
Jul 07 15:01:54.279855 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:54.278282    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f57ff8d78-dnmf6" event={"ID":"86273f20-948b-4ff4-a02f-007646e091d9","Type":"ContainerStarted","Data":"4a0e689de707199db8b98d0a761118c84add64d59809294abc378c2174229a3f"}
Jul 07 15:01:54.279855 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:54.278911    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f57ff8d78-dnmf6"
Jul 07 15:01:54.282221 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:54.282193    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f57ff8d78-dnmf6"
Jul 07 15:01:54.282472 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:54.282446    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qt79g" event={"ID":"56f88c5a-6833-4eab-b858-8c4b163fbdcb","Type":"ContainerStarted","Data":"078a794579dd1c34ee46202c6c09e656345d7419c6c975b5149d8004d80758dc"}
Jul 07 15:01:54.285490 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:54.285426    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6m26d" event={"ID":"c81a8d96-38ae-48b5-bd30-2711751dedef","Type":"ContainerStarted","Data":"5a9e207cc14e33b2d113f3783ae615382d85fed36f38803185f9154d983bac45"}
Jul 07 15:01:54.287789 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:54.287764    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rlrhq" event={"ID":"23819f39-1440-4b6c-90fa-6db19eb48b74","Type":"ContainerStarted","Data":"afc1e172493f7f2e46b7daee45cd29598cb096509bfeb5d50eb89ec895e6382f"}
Jul 07 15:01:54.291135 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:54.291028    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-pfl6c" podStartSLOduration=3.3983435699999998 podStartE2EDuration="13.29101112s" totalImagesPullingTime="9.89266755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:54.290346912 +0000 UTC m=+82.086883664" watchObservedRunningTime="2026-07-07 15:01:54.29101112 +0000 UTC m=+82.087547849"
Jul 07 15:01:54.326406 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:54.326292    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7497d66c94-q942f" podStartSLOduration=3.614734659 podStartE2EDuration="13.326270005s" totalImagesPullingTime="9.711535346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:54.326129424 +0000 UTC m=+82.122666169" watchObservedRunningTime="2026-07-07 15:01:54.326270005 +0000 UTC m=+82.122806736"
Jul 07 15:01:54.355930 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:54.355027    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f57ff8d78-dnmf6" podStartSLOduration=3.6319219350000003 podStartE2EDuration="13.355011109s" totalImagesPullingTime="9.723089174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:54.353084836 +0000 UTC m=+82.149621629" watchObservedRunningTime="2026-07-07 15:01:54.355011109 +0000 UTC m=+82.151547836"
Jul 07 15:01:54.382725 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:54.381167    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-qzhfn" podStartSLOduration=3.687216108 podStartE2EDuration="13.381150632s" totalImagesPullingTime="9.693934524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:54.379558583 +0000 UTC m=+82.176095312" watchObservedRunningTime="2026-07-07 15:01:54.381150632 +0000 UTC m=+82.177687339"
Jul 07 15:01:54.405918 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:54.405560    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qt79g" podStartSLOduration=49.099535867 podStartE2EDuration="59.405543s" totalImagesPullingTime="10.306007133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:00:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:54.404655469 +0000 UTC m=+82.201192220" watchObservedRunningTime="2026-07-07 15:01:54.405543 +0000 UTC m=+82.202079727"
Jul 07 15:01:55.295456 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:55.295414    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rlrhq" event={"ID":"23819f39-1440-4b6c-90fa-6db19eb48b74","Type":"ContainerStarted","Data":"0905517068e11b0cfc6010ba4e390a31dd124b5c027720465353778a386388a0"}
Jul 07 15:01:55.295944 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:55.295497    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rlrhq"
Jul 07 15:01:55.298280 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:55.298249    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc66794f-l88nv" event={"ID":"a2edf693-41a7-449b-80d7-fb05e08e0e78","Type":"ContainerStarted","Data":"476b85d509e0ba4198b87b7a3086860cdef7b72a1e5298761a8ced6c879b221f"}
Jul 07 15:01:55.320545 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:55.320494    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rlrhq" podStartSLOduration=37.317043196 podStartE2EDuration="47.320476653s" totalImagesPullingTime="10.003433457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:55.319665222 +0000 UTC m=+83.116201963" watchObservedRunningTime="2026-07-07 15:01:55.320476653 +0000 UTC m=+83.117013377"
Jul 07 15:01:55.347755 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:55.347690    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fdc66794f-l88nv" podStartSLOduration=4.395587707 podStartE2EDuration="14.347667663s" totalImagesPullingTime="9.952079956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:55.347108733 +0000 UTC m=+83.143645471" watchObservedRunningTime="2026-07-07 15:01:55.347667663 +0000 UTC m=+83.144204391"
Jul 07 15:01:57.309935 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:57.309875    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6m26d" event={"ID":"c81a8d96-38ae-48b5-bd30-2711751dedef","Type":"ContainerStarted","Data":"7bcbe5a4519eb47339ef41ac3d04c088ba70000881e32d38236524593b3f03be"}
Jul 07 15:01:57.336215 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:01:57.336161    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6m26d" podStartSLOduration=19.54235003 podStartE2EDuration="37.336144134s" totalImagesPullingTime="17.793794104s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:01:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:57.333870713 +0000 UTC m=+85.130407460" watchObservedRunningTime="2026-07-07 15:01:57.336144134 +0000 UTC m=+85.132680856"
Jul 07 15:02:00.185529 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:00.185495    2567 patch_prober.go:28] interesting pod/image-registry-6cfbcb4769-87kk4 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 07 15:02:00.185944 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:00.185550    2567 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4" podUID="d050a3c0-f99d-4e6d-99f3-03ce677e7fa0" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:02:05.217979 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:05.217948    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:02:05.305640 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:05.305606    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rlrhq"
Jul 07 15:02:07.175234 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.175069    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ksrg4"]
Jul 07 15:02:07.290976 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.290944    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:07.294220 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.294191    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 07 15:02:07.294871 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.294661    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 07 15:02:07.294871 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.294758    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 07 15:02:07.296982 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.296952    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 07 15:02:07.297166 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.297150    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5lxkv\""
Jul 07 15:02:07.379768 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.379335    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/13d53bdc-0638-415f-b8f3-ffb33edea025-root\") pod \"node-exporter-ksrg4\" (UID: \"13d53bdc-0638-415f-b8f3-ffb33edea025\") " pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:07.379768 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.379388    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/13d53bdc-0638-415f-b8f3-ffb33edea025-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ksrg4\" (UID: \"13d53bdc-0638-415f-b8f3-ffb33edea025\") " pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:07.379768 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.379420    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/13d53bdc-0638-415f-b8f3-ffb33edea025-sys\") pod \"node-exporter-ksrg4\" (UID: \"13d53bdc-0638-415f-b8f3-ffb33edea025\") " pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:07.379768 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.379475    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/13d53bdc-0638-415f-b8f3-ffb33edea025-node-exporter-tls\") pod \"node-exporter-ksrg4\" (UID: \"13d53bdc-0638-415f-b8f3-ffb33edea025\") " pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:07.379768 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.379503    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pndhp\" (UniqueName: \"kubernetes.io/projected/13d53bdc-0638-415f-b8f3-ffb33edea025-kube-api-access-pndhp\") pod \"node-exporter-ksrg4\" (UID: \"13d53bdc-0638-415f-b8f3-ffb33edea025\") " pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:07.379768 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.379529    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/13d53bdc-0638-415f-b8f3-ffb33edea025-node-exporter-wtmp\") pod \"node-exporter-ksrg4\" (UID: \"13d53bdc-0638-415f-b8f3-ffb33edea025\") " pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:07.379768 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.379561    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13d53bdc-0638-415f-b8f3-ffb33edea025-metrics-client-ca\") pod \"node-exporter-ksrg4\" (UID: \"13d53bdc-0638-415f-b8f3-ffb33edea025\") " pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:07.379768 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.379598    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/13d53bdc-0638-415f-b8f3-ffb33edea025-node-exporter-accelerators-collector-config\") pod \"node-exporter-ksrg4\" (UID: \"13d53bdc-0638-415f-b8f3-ffb33edea025\") " pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:07.379768 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.379642    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/13d53bdc-0638-415f-b8f3-ffb33edea025-node-exporter-textfile\") pod \"node-exporter-ksrg4\" (UID: \"13d53bdc-0638-415f-b8f3-ffb33edea025\") " pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:07.480323 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.480247    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/13d53bdc-0638-415f-b8f3-ffb33edea025-root\") pod \"node-exporter-ksrg4\" (UID: \"13d53bdc-0638-415f-b8f3-ffb33edea025\") " pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:07.480323 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.480301    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/13d53bdc-0638-415f-b8f3-ffb33edea025-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ksrg4\" (UID: \"13d53bdc-0638-415f-b8f3-ffb33edea025\") " pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:07.480493 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.480329    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/13d53bdc-0638-415f-b8f3-ffb33edea025-sys\") pod \"node-exporter-ksrg4\" (UID: \"13d53bdc-0638-415f-b8f3-ffb33edea025\") " pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:07.480493 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.480373    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/13d53bdc-0638-415f-b8f3-ffb33edea025-node-exporter-tls\") pod \"node-exporter-ksrg4\" (UID: \"13d53bdc-0638-415f-b8f3-ffb33edea025\") " pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:07.480493 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.480402    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pndhp\" (UniqueName: \"kubernetes.io/projected/13d53bdc-0638-415f-b8f3-ffb33edea025-kube-api-access-pndhp\") pod \"node-exporter-ksrg4\" (UID: \"13d53bdc-0638-415f-b8f3-ffb33edea025\") " pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:07.480493 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.480428    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/13d53bdc-0638-415f-b8f3-ffb33edea025-node-exporter-wtmp\") pod \"node-exporter-ksrg4\" (UID: \"13d53bdc-0638-415f-b8f3-ffb33edea025\") " pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:07.480493 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.480457    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13d53bdc-0638-415f-b8f3-ffb33edea025-metrics-client-ca\") pod \"node-exporter-ksrg4\" (UID: \"13d53bdc-0638-415f-b8f3-ffb33edea025\") " pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:07.480640 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.480495    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/13d53bdc-0638-415f-b8f3-ffb33edea025-node-exporter-accelerators-collector-config\") pod \"node-exporter-ksrg4\" (UID: \"13d53bdc-0638-415f-b8f3-ffb33edea025\") " pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:07.480640 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.480538    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/13d53bdc-0638-415f-b8f3-ffb33edea025-node-exporter-textfile\") pod \"node-exporter-ksrg4\" (UID: \"13d53bdc-0638-415f-b8f3-ffb33edea025\") " pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:07.480920 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.480903    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/13d53bdc-0638-415f-b8f3-ffb33edea025-node-exporter-textfile\") pod \"node-exporter-ksrg4\" (UID: \"13d53bdc-0638-415f-b8f3-ffb33edea025\") " pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:07.481002 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.480976    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/13d53bdc-0638-415f-b8f3-ffb33edea025-root\") pod \"node-exporter-ksrg4\" (UID: \"13d53bdc-0638-415f-b8f3-ffb33edea025\") " pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:07.481954 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.481408    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/13d53bdc-0638-415f-b8f3-ffb33edea025-sys\") pod \"node-exporter-ksrg4\" (UID: \"13d53bdc-0638-415f-b8f3-ffb33edea025\") " pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:07.481954 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:02:07.481465    2567 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 07 15:02:07.481954 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:02:07.481531    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13d53bdc-0638-415f-b8f3-ffb33edea025-node-exporter-tls podName:13d53bdc-0638-415f-b8f3-ffb33edea025 nodeName:}" failed. No retries permitted until 2026-07-07 15:02:07.981509022 +0000 UTC m=+95.778045751 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/13d53bdc-0638-415f-b8f3-ffb33edea025-node-exporter-tls") pod "node-exporter-ksrg4" (UID: "13d53bdc-0638-415f-b8f3-ffb33edea025") : secret "node-exporter-tls" not found
Jul 07 15:02:07.481954 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.481823    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/13d53bdc-0638-415f-b8f3-ffb33edea025-node-exporter-wtmp\") pod \"node-exporter-ksrg4\" (UID: \"13d53bdc-0638-415f-b8f3-ffb33edea025\") " pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:07.481954 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.481902    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13d53bdc-0638-415f-b8f3-ffb33edea025-metrics-client-ca\") pod \"node-exporter-ksrg4\" (UID: \"13d53bdc-0638-415f-b8f3-ffb33edea025\") " pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:07.482383 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.482356    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/13d53bdc-0638-415f-b8f3-ffb33edea025-node-exporter-accelerators-collector-config\") pod \"node-exporter-ksrg4\" (UID: \"13d53bdc-0638-415f-b8f3-ffb33edea025\") " pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:07.486371 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.486352    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/13d53bdc-0638-415f-b8f3-ffb33edea025-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ksrg4\" (UID: \"13d53bdc-0638-415f-b8f3-ffb33edea025\") " pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:07.498550 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.498526    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pndhp\" (UniqueName: \"kubernetes.io/projected/13d53bdc-0638-415f-b8f3-ffb33edea025-kube-api-access-pndhp\") pod \"node-exporter-ksrg4\" (UID: \"13d53bdc-0638-415f-b8f3-ffb33edea025\") " pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:07.984771 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.984733    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/13d53bdc-0638-415f-b8f3-ffb33edea025-node-exporter-tls\") pod \"node-exporter-ksrg4\" (UID: \"13d53bdc-0638-415f-b8f3-ffb33edea025\") " pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:07.987668 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:07.987640    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/13d53bdc-0638-415f-b8f3-ffb33edea025-node-exporter-tls\") pod \"node-exporter-ksrg4\" (UID: \"13d53bdc-0638-415f-b8f3-ffb33edea025\") " pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:08.207302 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:08.207266    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ksrg4"
Jul 07 15:02:08.419960 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:08.419925    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6cfbcb4769-87kk4"]
Jul 07 15:02:10.162993 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:10.162963    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9gmwd"
Jul 07 15:02:10.741995 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:02:10.741962    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13d53bdc_0638_415f_b8f3_ffb33edea025.slice/crio-fc6685382d754a3680781f4602ee0d09dad6a08467f7ddd331a0b4d326f43360 WatchSource:0}: Error finding container fc6685382d754a3680781f4602ee0d09dad6a08467f7ddd331a0b4d326f43360: Status 404 returned error can't find the container with id fc6685382d754a3680781f4602ee0d09dad6a08467f7ddd331a0b4d326f43360
Jul 07 15:02:11.373594 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:11.373487    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-chvmp" event={"ID":"acef1cce-5704-4d95-9418-8e87b81831bb","Type":"ContainerStarted","Data":"ae04f4feaefa60db065cb1e25aa2f1279d4f80fd496c2a91526845e9a4d2dbf5"}
Jul 07 15:02:11.374080 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:11.373601    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-chvmp"
Jul 07 15:02:11.375254 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:11.375223    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ksrg4" event={"ID":"13d53bdc-0638-415f-b8f3-ffb33edea025","Type":"ContainerStarted","Data":"fc6685382d754a3680781f4602ee0d09dad6a08467f7ddd331a0b4d326f43360"}
Jul 07 15:02:11.375439 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:11.375413    2567 patch_prober.go:28] interesting pod/downloads-685bfb5f64-chvmp container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.25:8080/\": dial tcp 10.132.0.25:8080: connect: connection refused" start-of-body=
Jul 07 15:02:11.375527 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:11.375458    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-685bfb5f64-chvmp" podUID="acef1cce-5704-4d95-9418-8e87b81831bb" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.25:8080/\": dial tcp 10.132.0.25:8080: connect: connection refused"
Jul 07 15:02:11.399264 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:11.399191    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-chvmp" podStartSLOduration=2.674063676 podStartE2EDuration="26.399145155s" totalImagesPullingTime="23.725081479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:02:11.397454858 +0000 UTC m=+99.193991615" watchObservedRunningTime="2026-07-07 15:02:11.399145155 +0000 UTC m=+99.195681882"
Jul 07 15:02:12.381949 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:12.381861    2567 generic.go:356] "Generic (PLEG): container finished" podID="13d53bdc-0638-415f-b8f3-ffb33edea025" containerID="03e476ad8fa0c3671e22495bbfae8c700cd3f86f8c25dc5796f0cb9e0c0bfb8d" exitCode=0
Jul 07 15:02:12.381949 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:12.381930    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ksrg4" event={"ID":"13d53bdc-0638-415f-b8f3-ffb33edea025","Type":"ContainerDied","Data":"03e476ad8fa0c3671e22495bbfae8c700cd3f86f8c25dc5796f0cb9e0c0bfb8d"}
Jul 07 15:02:12.395194 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:12.395163    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-chvmp"
Jul 07 15:02:13.389868 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:13.389811    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ksrg4" event={"ID":"13d53bdc-0638-415f-b8f3-ffb33edea025","Type":"ContainerStarted","Data":"742e4593b25b28de3a866244f6aeaa34edb4104738e7298719516237b6cb439b"}
Jul 07 15:02:13.390373 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:13.389898    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ksrg4" event={"ID":"13d53bdc-0638-415f-b8f3-ffb33edea025","Type":"ContainerStarted","Data":"d2cfafd4286de24f920a1833c6f108cb5358f1eabe1f3befb8a169da6450483d"}
Jul 07 15:02:13.423504 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:13.423428    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ksrg4" podStartSLOduration=5.209357615 podStartE2EDuration="6.423404108s" totalImagesPullingTime="1.101157301s" totalInitContainerRuntime="112.889192ms" isStatefulPod=true podCreationTimestamp="2026-07-07 15:02:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:02:13.421358049 +0000 UTC m=+101.217894796" watchObservedRunningTime="2026-07-07 15:02:13.423404108 +0000 UTC m=+101.219940857"
Jul 07 15:02:24.438104 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:24.438065    2567 generic.go:356] "Generic (PLEG): container finished" podID="73d589b0-98d3-40b3-8179-27d4ff10efbf" containerID="ed859ca60631a73c08aeb86df55345f8b12987bfa3ef4025944b8395bd9e5c0e" exitCode=0
Jul 07 15:02:24.438491 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:24.438137    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-cz626" event={"ID":"73d589b0-98d3-40b3-8179-27d4ff10efbf","Type":"ContainerDied","Data":"ed859ca60631a73c08aeb86df55345f8b12987bfa3ef4025944b8395bd9e5c0e"}
Jul 07 15:02:24.438491 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:24.438447    2567 scope.go:117] "RemoveContainer" containerID="ed859ca60631a73c08aeb86df55345f8b12987bfa3ef4025944b8395bd9e5c0e"
Jul 07 15:02:25.443817 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:25.443783    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-cz626" event={"ID":"73d589b0-98d3-40b3-8179-27d4ff10efbf","Type":"ContainerStarted","Data":"b089c60ec4874263b601c2397fecaaafdc27a47bfa975e0c176cd0a382f46600"}
Jul 07 15:02:30.463432 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:30.463398    2567 generic.go:356] "Generic (PLEG): container finished" podID="422c8ccc-828c-4491-a88b-76cd6973e056" containerID="34b537d54a79c26ed1da1691eb062019183657f19040914cd751d6bf4ed7b0a8" exitCode=0
Jul 07 15:02:30.463900 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:30.463505    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-d5g6n" event={"ID":"422c8ccc-828c-4491-a88b-76cd6973e056","Type":"ContainerDied","Data":"34b537d54a79c26ed1da1691eb062019183657f19040914cd751d6bf4ed7b0a8"}
Jul 07 15:02:30.463900 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:30.463894    2567 scope.go:117] "RemoveContainer" containerID="34b537d54a79c26ed1da1691eb062019183657f19040914cd751d6bf4ed7b0a8"
Jul 07 15:02:31.469134 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:31.469096    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-d5g6n" event={"ID":"422c8ccc-828c-4491-a88b-76cd6973e056","Type":"ContainerStarted","Data":"15109c9e1b40b8e4059c9f6e5f2225e6c024d4e8a88fb858e554f9af17ad268b"}
Jul 07 15:02:33.445483 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:33.445422    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4" podUID="d050a3c0-f99d-4e6d-99f3-03ce677e7fa0" containerName="registry" containerID="cri-o://6c56dac1deb9c0aff7f0f1e9aa636af608c719046d571d9e70860e7111f85f37" gracePeriod=30
Jul 07 15:02:33.717490 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:33.717420    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:02:33.822128 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:33.822090    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-tls\") pod \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") "
Jul 07 15:02:33.822348 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:33.822152    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9lb6\" (UniqueName: \"kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-kube-api-access-n9lb6\") pod \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") "
Jul 07 15:02:33.822348 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:33.822179    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-image-registry-private-configuration\") pod \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") "
Jul 07 15:02:33.822348 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:33.822251    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-bound-sa-token\") pod \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") "
Jul 07 15:02:33.822348 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:33.822311    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-installation-pull-secrets\") pod \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") "
Jul 07 15:02:33.822557 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:33.822359    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-ca-trust-extracted\") pod \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") "
Jul 07 15:02:33.822557 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:33.822383    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-certificates\") pod \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") "
Jul 07 15:02:33.822557 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:33.822438    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-trusted-ca\") pod \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\" (UID: \"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0\") "
Jul 07 15:02:33.823138 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:33.823018    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d050a3c0-f99d-4e6d-99f3-03ce677e7fa0" (UID: "d050a3c0-f99d-4e6d-99f3-03ce677e7fa0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:02:33.823260 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:33.823195    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d050a3c0-f99d-4e6d-99f3-03ce677e7fa0" (UID: "d050a3c0-f99d-4e6d-99f3-03ce677e7fa0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:02:33.824970 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:33.824935    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d050a3c0-f99d-4e6d-99f3-03ce677e7fa0" (UID: "d050a3c0-f99d-4e6d-99f3-03ce677e7fa0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:02:33.825088 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:33.824994    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-kube-api-access-n9lb6" (OuterVolumeSpecName: "kube-api-access-n9lb6") pod "d050a3c0-f99d-4e6d-99f3-03ce677e7fa0" (UID: "d050a3c0-f99d-4e6d-99f3-03ce677e7fa0"). InnerVolumeSpecName "kube-api-access-n9lb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:02:33.825088 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:33.825001    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d050a3c0-f99d-4e6d-99f3-03ce677e7fa0" (UID: "d050a3c0-f99d-4e6d-99f3-03ce677e7fa0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:02:33.825088 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:33.825027    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d050a3c0-f99d-4e6d-99f3-03ce677e7fa0" (UID: "d050a3c0-f99d-4e6d-99f3-03ce677e7fa0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:02:33.825088 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:33.825079    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d050a3c0-f99d-4e6d-99f3-03ce677e7fa0" (UID: "d050a3c0-f99d-4e6d-99f3-03ce677e7fa0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:02:33.830620 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:33.830595    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d050a3c0-f99d-4e6d-99f3-03ce677e7fa0" (UID: "d050a3c0-f99d-4e6d-99f3-03ce677e7fa0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:02:33.923254 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:33.923216    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-trusted-ca\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:02:33.923254 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:33.923248    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-tls\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:02:33.923254 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:33.923259    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9lb6\" (UniqueName: \"kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-kube-api-access-n9lb6\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:02:33.923508 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:33.923273    2567 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-image-registry-private-configuration\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:02:33.923508 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:33.923286    2567 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-bound-sa-token\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:02:33.923508 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:33.923297    2567 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-installation-pull-secrets\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:02:33.923508 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:33.923379    2567 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-ca-trust-extracted\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:02:33.923508 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:33.923393    2567 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0-registry-certificates\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:02:34.483457 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:34.483423    2567 generic.go:356] "Generic (PLEG): container finished" podID="d050a3c0-f99d-4e6d-99f3-03ce677e7fa0" containerID="6c56dac1deb9c0aff7f0f1e9aa636af608c719046d571d9e70860e7111f85f37" exitCode=0
Jul 07 15:02:34.483457 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:34.483464    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4"
Jul 07 15:02:34.483992 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:34.483466    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4" event={"ID":"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0","Type":"ContainerDied","Data":"6c56dac1deb9c0aff7f0f1e9aa636af608c719046d571d9e70860e7111f85f37"}
Jul 07 15:02:34.483992 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:34.483594    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cfbcb4769-87kk4" event={"ID":"d050a3c0-f99d-4e6d-99f3-03ce677e7fa0","Type":"ContainerDied","Data":"6b96dfa547dadd642ca42247b50e5662003875fd6ade50d5d90ea16b9a2fa700"}
Jul 07 15:02:34.483992 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:34.483618    2567 scope.go:117] "RemoveContainer" containerID="6c56dac1deb9c0aff7f0f1e9aa636af608c719046d571d9e70860e7111f85f37"
Jul 07 15:02:34.492590 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:34.492568    2567 scope.go:117] "RemoveContainer" containerID="6c56dac1deb9c0aff7f0f1e9aa636af608c719046d571d9e70860e7111f85f37"
Jul 07 15:02:34.492884 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:02:34.492865    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c56dac1deb9c0aff7f0f1e9aa636af608c719046d571d9e70860e7111f85f37\": container with ID starting with 6c56dac1deb9c0aff7f0f1e9aa636af608c719046d571d9e70860e7111f85f37 not found: ID does not exist" containerID="6c56dac1deb9c0aff7f0f1e9aa636af608c719046d571d9e70860e7111f85f37"
Jul 07 15:02:34.492961 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:34.492892    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c56dac1deb9c0aff7f0f1e9aa636af608c719046d571d9e70860e7111f85f37"} err="failed to get container status \"6c56dac1deb9c0aff7f0f1e9aa636af608c719046d571d9e70860e7111f85f37\": rpc error: code = NotFound desc = could not find container \"6c56dac1deb9c0aff7f0f1e9aa636af608c719046d571d9e70860e7111f85f37\": container with ID starting with 6c56dac1deb9c0aff7f0f1e9aa636af608c719046d571d9e70860e7111f85f37 not found: ID does not exist"
Jul 07 15:02:34.515479 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:34.515453    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6cfbcb4769-87kk4"]
Jul 07 15:02:34.522369 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:34.522348    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6cfbcb4769-87kk4"]
Jul 07 15:02:34.776948 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:34.776873    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d050a3c0-f99d-4e6d-99f3-03ce677e7fa0" path="/var/lib/kubelet/pods/d050a3c0-f99d-4e6d-99f3-03ce677e7fa0/volumes"
Jul 07 15:02:35.494560 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:35.494446    2567 generic.go:356] "Generic (PLEG): container finished" podID="719d2a5c-4213-4e2a-bce0-9b29e0ad02c0" containerID="22d761a1dc7acad0de92aea7bd050ef2da017c002aee52b238cc5da074c0788a" exitCode=0
Jul 07 15:02:35.494560 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:35.494517    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tfz7j" event={"ID":"719d2a5c-4213-4e2a-bce0-9b29e0ad02c0","Type":"ContainerDied","Data":"22d761a1dc7acad0de92aea7bd050ef2da017c002aee52b238cc5da074c0788a"}
Jul 07 15:02:35.495101 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:35.494912    2567 scope.go:117] "RemoveContainer" containerID="22d761a1dc7acad0de92aea7bd050ef2da017c002aee52b238cc5da074c0788a"
Jul 07 15:02:36.500527 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:02:36.500493    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-tfz7j" event={"ID":"719d2a5c-4213-4e2a-bce0-9b29e0ad02c0","Type":"ContainerStarted","Data":"3a26b97b72918022c1c7ccb175757fc5d840586b3a3503ac02c43c21acfdb3d5"}
Jul 07 15:05:32.638892 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:32.638864    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5r9rp_1c51cd93-b0ea-4aa1-8678-a4c833b2d37e/console-operator/1.log"
Jul 07 15:05:32.642195 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:32.642172    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5r9rp_1c51cd93-b0ea-4aa1-8678-a4c833b2d37e/console-operator/1.log"
Jul 07 15:05:32.651916 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:32.651892    2567 kubelet.go:1648] "Image garbage collection succeeded"
Jul 07 15:05:52.010492 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.010463    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-845dc9964f-k4w2t"]
Jul 07 15:05:52.011903 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.010755    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d050a3c0-f99d-4e6d-99f3-03ce677e7fa0" containerName="registry"
Jul 07 15:05:52.011903 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.010766    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d050a3c0-f99d-4e6d-99f3-03ce677e7fa0" containerName="registry"
Jul 07 15:05:52.011903 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.010824    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d050a3c0-f99d-4e6d-99f3-03ce677e7fa0" containerName="registry"
Jul 07 15:05:52.012666 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.012651    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-k4w2t"
Jul 07 15:05:52.017454 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.017415    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 07 15:05:52.017728 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.017707    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7xnv8\""
Jul 07 15:05:52.017819 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.017748    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 07 15:05:52.018007 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.017986    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 07 15:05:52.018362 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.018346    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 07 15:05:52.049633 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.049601    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-845dc9964f-k4w2t"]
Jul 07 15:05:52.104598 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.104562    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/be71a3b9-203e-4c8a-bd44-9994eb9813c8-tmp\") pod \"opendatahub-operator-controller-manager-845dc9964f-k4w2t\" (UID: \"be71a3b9-203e-4c8a-bd44-9994eb9813c8\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-k4w2t"
Jul 07 15:05:52.104774 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.104605    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be71a3b9-203e-4c8a-bd44-9994eb9813c8-webhook-cert\") pod \"opendatahub-operator-controller-manager-845dc9964f-k4w2t\" (UID: \"be71a3b9-203e-4c8a-bd44-9994eb9813c8\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-k4w2t"
Jul 07 15:05:52.104774 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.104624    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2ds7\" (UniqueName: \"kubernetes.io/projected/be71a3b9-203e-4c8a-bd44-9994eb9813c8-kube-api-access-t2ds7\") pod \"opendatahub-operator-controller-manager-845dc9964f-k4w2t\" (UID: \"be71a3b9-203e-4c8a-bd44-9994eb9813c8\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-k4w2t"
Jul 07 15:05:52.104774 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.104651    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/be71a3b9-203e-4c8a-bd44-9994eb9813c8-manifests\") pod \"opendatahub-operator-controller-manager-845dc9964f-k4w2t\" (UID: \"be71a3b9-203e-4c8a-bd44-9994eb9813c8\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-k4w2t"
Jul 07 15:05:52.104774 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.104710    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/be71a3b9-203e-4c8a-bd44-9994eb9813c8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-845dc9964f-k4w2t\" (UID: \"be71a3b9-203e-4c8a-bd44-9994eb9813c8\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-k4w2t"
Jul 07 15:05:52.206163 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.206123    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be71a3b9-203e-4c8a-bd44-9994eb9813c8-webhook-cert\") pod \"opendatahub-operator-controller-manager-845dc9964f-k4w2t\" (UID: \"be71a3b9-203e-4c8a-bd44-9994eb9813c8\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-k4w2t"
Jul 07 15:05:52.206163 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.206170    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2ds7\" (UniqueName: \"kubernetes.io/projected/be71a3b9-203e-4c8a-bd44-9994eb9813c8-kube-api-access-t2ds7\") pod \"opendatahub-operator-controller-manager-845dc9964f-k4w2t\" (UID: \"be71a3b9-203e-4c8a-bd44-9994eb9813c8\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-k4w2t"
Jul 07 15:05:52.206450 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.206208    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/be71a3b9-203e-4c8a-bd44-9994eb9813c8-manifests\") pod \"opendatahub-operator-controller-manager-845dc9964f-k4w2t\" (UID: \"be71a3b9-203e-4c8a-bd44-9994eb9813c8\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-k4w2t"
Jul 07 15:05:52.206450 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.206237    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/be71a3b9-203e-4c8a-bd44-9994eb9813c8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-845dc9964f-k4w2t\" (UID: \"be71a3b9-203e-4c8a-bd44-9994eb9813c8\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-k4w2t"
Jul 07 15:05:52.206450 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.206320    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/be71a3b9-203e-4c8a-bd44-9994eb9813c8-tmp\") pod \"opendatahub-operator-controller-manager-845dc9964f-k4w2t\" (UID: \"be71a3b9-203e-4c8a-bd44-9994eb9813c8\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-k4w2t"
Jul 07 15:05:52.206649 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.206625    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/be71a3b9-203e-4c8a-bd44-9994eb9813c8-manifests\") pod \"opendatahub-operator-controller-manager-845dc9964f-k4w2t\" (UID: \"be71a3b9-203e-4c8a-bd44-9994eb9813c8\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-k4w2t"
Jul 07 15:05:52.208820 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.208790    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/be71a3b9-203e-4c8a-bd44-9994eb9813c8-tmp\") pod \"opendatahub-operator-controller-manager-845dc9964f-k4w2t\" (UID: \"be71a3b9-203e-4c8a-bd44-9994eb9813c8\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-k4w2t"
Jul 07 15:05:52.208987 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.208903    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be71a3b9-203e-4c8a-bd44-9994eb9813c8-webhook-cert\") pod \"opendatahub-operator-controller-manager-845dc9964f-k4w2t\" (UID: \"be71a3b9-203e-4c8a-bd44-9994eb9813c8\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-k4w2t"
Jul 07 15:05:52.209103 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.209084    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/be71a3b9-203e-4c8a-bd44-9994eb9813c8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-845dc9964f-k4w2t\" (UID: \"be71a3b9-203e-4c8a-bd44-9994eb9813c8\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-k4w2t"
Jul 07 15:05:52.228229 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.228193    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2ds7\" (UniqueName: \"kubernetes.io/projected/be71a3b9-203e-4c8a-bd44-9994eb9813c8-kube-api-access-t2ds7\") pod \"opendatahub-operator-controller-manager-845dc9964f-k4w2t\" (UID: \"be71a3b9-203e-4c8a-bd44-9994eb9813c8\") " pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-k4w2t"
Jul 07 15:05:52.323461 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.323417    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-k4w2t"
Jul 07 15:05:52.470727 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.470674    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-845dc9964f-k4w2t"]
Jul 07 15:05:52.473441 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:05:52.473412    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe71a3b9_203e_4c8a_bd44_9994eb9813c8.slice/crio-029a34805b2296d7b6e8907baf59170b364aab7dd3dbd2c5d1d91e0b8e063a7a WatchSource:0}: Error finding container 029a34805b2296d7b6e8907baf59170b364aab7dd3dbd2c5d1d91e0b8e063a7a: Status 404 returned error can't find the container with id 029a34805b2296d7b6e8907baf59170b364aab7dd3dbd2c5d1d91e0b8e063a7a
Jul 07 15:05:52.475419 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:52.475399    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 15:05:53.248207 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:53.248172    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-k4w2t" event={"ID":"be71a3b9-203e-4c8a-bd44-9994eb9813c8","Type":"ContainerStarted","Data":"029a34805b2296d7b6e8907baf59170b364aab7dd3dbd2c5d1d91e0b8e063a7a"}
Jul 07 15:05:56.264101 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:56.264060    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-k4w2t" event={"ID":"be71a3b9-203e-4c8a-bd44-9994eb9813c8","Type":"ContainerStarted","Data":"c61220c5a594888481439212e4ad8296a2f13f8e4931c6191b885ef06a378409"}
Jul 07 15:05:57.269818 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:57.269713    2567 generic.go:356] "Generic (PLEG): container finished" podID="be71a3b9-203e-4c8a-bd44-9994eb9813c8" containerID="c61220c5a594888481439212e4ad8296a2f13f8e4931c6191b885ef06a378409" exitCode=0
Jul 07 15:05:57.269818 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:57.269808    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-k4w2t" event={"ID":"be71a3b9-203e-4c8a-bd44-9994eb9813c8","Type":"ContainerDied","Data":"c61220c5a594888481439212e4ad8296a2f13f8e4931c6191b885ef06a378409"}
Jul 07 15:05:58.277323 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:58.277279    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-k4w2t" event={"ID":"be71a3b9-203e-4c8a-bd44-9994eb9813c8","Type":"ContainerStarted","Data":"b30441a4a380a003c71cf19500bd777975928bf45cfc014d65759a4cef9ae7dd"}
Jul 07 15:05:58.277739 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:05:58.277427    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-k4w2t"
Jul 07 15:06:09.284412 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:09.284382    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-k4w2t"
Jul 07 15:06:09.318173 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:09.318121    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-845dc9964f-k4w2t" podStartSLOduration=13.203458772 podStartE2EDuration="18.318102424s" totalImagesPullingTime="4.110518164s" totalInitContainerRuntime="1.004125488s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:05:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:05:58.314733512 +0000 UTC m=+326.111270239" watchObservedRunningTime="2026-07-07 15:06:09.318102424 +0000 UTC m=+337.114639151"
Jul 07 15:06:12.428808 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:12.428765    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-8697c578dd-klxhp"]
Jul 07 15:06:12.439674 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:12.439631    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8697c578dd-klxhp"
Jul 07 15:06:12.444656 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:12.444627    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-rx945\""
Jul 07 15:06:12.444886 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:12.444865    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 07 15:06:12.444951 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:12.444899    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 07 15:06:12.445794 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:12.445756    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8697c578dd-klxhp"]
Jul 07 15:06:12.469586 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:12.469550    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2338f6c-01a4-4b70-81c8-79bb3caecad7-tmp\") pod \"kube-auth-proxy-8697c578dd-klxhp\" (UID: \"c2338f6c-01a4-4b70-81c8-79bb3caecad7\") " pod="openshift-ingress/kube-auth-proxy-8697c578dd-klxhp"
Jul 07 15:06:12.469782 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:12.469609    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz2fg\" (UniqueName: \"kubernetes.io/projected/c2338f6c-01a4-4b70-81c8-79bb3caecad7-kube-api-access-hz2fg\") pod \"kube-auth-proxy-8697c578dd-klxhp\" (UID: \"c2338f6c-01a4-4b70-81c8-79bb3caecad7\") " pod="openshift-ingress/kube-auth-proxy-8697c578dd-klxhp"
Jul 07 15:06:12.469782 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:12.469694    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2338f6c-01a4-4b70-81c8-79bb3caecad7-tls-certs\") pod \"kube-auth-proxy-8697c578dd-klxhp\" (UID: \"c2338f6c-01a4-4b70-81c8-79bb3caecad7\") " pod="openshift-ingress/kube-auth-proxy-8697c578dd-klxhp"
Jul 07 15:06:12.571190 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:12.571075    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2338f6c-01a4-4b70-81c8-79bb3caecad7-tls-certs\") pod \"kube-auth-proxy-8697c578dd-klxhp\" (UID: \"c2338f6c-01a4-4b70-81c8-79bb3caecad7\") " pod="openshift-ingress/kube-auth-proxy-8697c578dd-klxhp"
Jul 07 15:06:12.571398 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:12.571258    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2338f6c-01a4-4b70-81c8-79bb3caecad7-tmp\") pod \"kube-auth-proxy-8697c578dd-klxhp\" (UID: \"c2338f6c-01a4-4b70-81c8-79bb3caecad7\") " pod="openshift-ingress/kube-auth-proxy-8697c578dd-klxhp"
Jul 07 15:06:12.571398 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:12.571293    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz2fg\" (UniqueName: \"kubernetes.io/projected/c2338f6c-01a4-4b70-81c8-79bb3caecad7-kube-api-access-hz2fg\") pod \"kube-auth-proxy-8697c578dd-klxhp\" (UID: \"c2338f6c-01a4-4b70-81c8-79bb3caecad7\") " pod="openshift-ingress/kube-auth-proxy-8697c578dd-klxhp"
Jul 07 15:06:12.573518 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:12.573499    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2338f6c-01a4-4b70-81c8-79bb3caecad7-tmp\") pod \"kube-auth-proxy-8697c578dd-klxhp\" (UID: \"c2338f6c-01a4-4b70-81c8-79bb3caecad7\") " pod="openshift-ingress/kube-auth-proxy-8697c578dd-klxhp"
Jul 07 15:06:12.573657 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:12.573636    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2338f6c-01a4-4b70-81c8-79bb3caecad7-tls-certs\") pod \"kube-auth-proxy-8697c578dd-klxhp\" (UID: \"c2338f6c-01a4-4b70-81c8-79bb3caecad7\") " pod="openshift-ingress/kube-auth-proxy-8697c578dd-klxhp"
Jul 07 15:06:12.588100 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:12.588063    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz2fg\" (UniqueName: \"kubernetes.io/projected/c2338f6c-01a4-4b70-81c8-79bb3caecad7-kube-api-access-hz2fg\") pod \"kube-auth-proxy-8697c578dd-klxhp\" (UID: \"c2338f6c-01a4-4b70-81c8-79bb3caecad7\") " pod="openshift-ingress/kube-auth-proxy-8697c578dd-klxhp"
Jul 07 15:06:12.751424 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:12.751342    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8697c578dd-klxhp"
Jul 07 15:06:12.863935 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:12.863895    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7r9lh"]
Jul 07 15:06:12.875618 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:12.875577    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7r9lh"]
Jul 07 15:06:12.875924 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:12.875900    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-7r9lh"
Jul 07 15:06:12.879349 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:12.879329    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 07 15:06:12.879349 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:12.879341    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-nnftm\""
Jul 07 15:06:12.910006 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:12.909970    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8697c578dd-klxhp"]
Jul 07 15:06:12.913088 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:06:12.913062    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2338f6c_01a4_4b70_81c8_79bb3caecad7.slice/crio-ba4362f7cedba7c2c003561444cc19b16fa9f4710607fc032e9d9cb555d3d05a WatchSource:0}: Error finding container ba4362f7cedba7c2c003561444cc19b16fa9f4710607fc032e9d9cb555d3d05a: Status 404 returned error can't find the container with id ba4362f7cedba7c2c003561444cc19b16fa9f4710607fc032e9d9cb555d3d05a
Jul 07 15:06:12.975396 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:12.975360    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/17173b3e-ac3b-4761-892d-be6bb3a484c3-cert\") pod \"odh-model-controller-7998c67dc4-7r9lh\" (UID: \"17173b3e-ac3b-4761-892d-be6bb3a484c3\") " pod="opendatahub/odh-model-controller-7998c67dc4-7r9lh"
Jul 07 15:06:12.975568 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:12.975431    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bk6j\" (UniqueName: \"kubernetes.io/projected/17173b3e-ac3b-4761-892d-be6bb3a484c3-kube-api-access-5bk6j\") pod \"odh-model-controller-7998c67dc4-7r9lh\" (UID: \"17173b3e-ac3b-4761-892d-be6bb3a484c3\") " pod="opendatahub/odh-model-controller-7998c67dc4-7r9lh"
Jul 07 15:06:13.076319 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:13.076281    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/17173b3e-ac3b-4761-892d-be6bb3a484c3-cert\") pod \"odh-model-controller-7998c67dc4-7r9lh\" (UID: \"17173b3e-ac3b-4761-892d-be6bb3a484c3\") " pod="opendatahub/odh-model-controller-7998c67dc4-7r9lh"
Jul 07 15:06:13.076516 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:13.076343    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bk6j\" (UniqueName: \"kubernetes.io/projected/17173b3e-ac3b-4761-892d-be6bb3a484c3-kube-api-access-5bk6j\") pod \"odh-model-controller-7998c67dc4-7r9lh\" (UID: \"17173b3e-ac3b-4761-892d-be6bb3a484c3\") " pod="opendatahub/odh-model-controller-7998c67dc4-7r9lh"
Jul 07 15:06:13.076516 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:06:13.076440    2567 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 07 15:06:13.076516 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:06:13.076517    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17173b3e-ac3b-4761-892d-be6bb3a484c3-cert podName:17173b3e-ac3b-4761-892d-be6bb3a484c3 nodeName:}" failed. No retries permitted until 2026-07-07 15:06:13.576500508 +0000 UTC m=+341.373037214 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/17173b3e-ac3b-4761-892d-be6bb3a484c3-cert") pod "odh-model-controller-7998c67dc4-7r9lh" (UID: "17173b3e-ac3b-4761-892d-be6bb3a484c3") : secret "odh-model-controller-webhook-cert" not found
Jul 07 15:06:13.090556 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:13.090521    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bk6j\" (UniqueName: \"kubernetes.io/projected/17173b3e-ac3b-4761-892d-be6bb3a484c3-kube-api-access-5bk6j\") pod \"odh-model-controller-7998c67dc4-7r9lh\" (UID: \"17173b3e-ac3b-4761-892d-be6bb3a484c3\") " pod="opendatahub/odh-model-controller-7998c67dc4-7r9lh"
Jul 07 15:06:13.344189 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:13.344090    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8697c578dd-klxhp" event={"ID":"c2338f6c-01a4-4b70-81c8-79bb3caecad7","Type":"ContainerStarted","Data":"ba4362f7cedba7c2c003561444cc19b16fa9f4710607fc032e9d9cb555d3d05a"}
Jul 07 15:06:13.580048 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:13.580009    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/17173b3e-ac3b-4761-892d-be6bb3a484c3-cert\") pod \"odh-model-controller-7998c67dc4-7r9lh\" (UID: \"17173b3e-ac3b-4761-892d-be6bb3a484c3\") " pod="opendatahub/odh-model-controller-7998c67dc4-7r9lh"
Jul 07 15:06:13.582739 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:13.582716    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/17173b3e-ac3b-4761-892d-be6bb3a484c3-cert\") pod \"odh-model-controller-7998c67dc4-7r9lh\" (UID: \"17173b3e-ac3b-4761-892d-be6bb3a484c3\") " pod="opendatahub/odh-model-controller-7998c67dc4-7r9lh"
Jul 07 15:06:13.788147 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:13.788055    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-7r9lh"
Jul 07 15:06:13.882739 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:13.882704    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5f6g2"]
Jul 07 15:06:13.886748 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:13.886680    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-5f6g2"
Jul 07 15:06:13.890003 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:13.889906    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 07 15:06:13.900904 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:13.900805    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5f6g2"]
Jul 07 15:06:13.910891 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:13.910139    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"]
Jul 07 15:06:13.915392 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:13.915369    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"
Jul 07 15:06:13.921954 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:13.920944    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 07 15:06:13.921954 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:13.921175    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 07 15:06:13.921954 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:13.921375    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-f6f58\""
Jul 07 15:06:13.921954 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:13.921579    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 07 15:06:13.921954 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:13.921780    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 07 15:06:13.936069 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:13.934009    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"]
Jul 07 15:06:13.982526 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:13.982487    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k9gz\" (UniqueName: \"kubernetes.io/projected/f8aeca71-6370-4687-8796-6bbe61f3bc92-kube-api-access-8k9gz\") pod \"model-serving-api-54d89bd5fc-5f6g2\" (UID: \"f8aeca71-6370-4687-8796-6bbe61f3bc92\") " pod="opendatahub/model-serving-api-54d89bd5fc-5f6g2"
Jul 07 15:06:13.982716 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:13.982536    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-local-certs\") pod \"istiod-openshift-gateway-7574459656-5vdc7\" (UID: \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"
Jul 07 15:06:13.982716 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:13.982567    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v65r4\" (UniqueName: \"kubernetes.io/projected/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-kube-api-access-v65r4\") pod \"istiod-openshift-gateway-7574459656-5vdc7\" (UID: \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"
Jul 07 15:06:13.982716 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:13.982590    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8aeca71-6370-4687-8796-6bbe61f3bc92-tls-certs\") pod \"model-serving-api-54d89bd5fc-5f6g2\" (UID: \"f8aeca71-6370-4687-8796-6bbe61f3bc92\") " pod="opendatahub/model-serving-api-54d89bd5fc-5f6g2"
Jul 07 15:06:13.982716 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:13.982604    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-cacerts\") pod \"istiod-openshift-gateway-7574459656-5vdc7\" (UID: \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"
Jul 07 15:06:13.982716 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:13.982620    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-5vdc7\" (UID: \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"
Jul 07 15:06:13.982716 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:13.982682    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-5vdc7\" (UID: \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"
Jul 07 15:06:13.983073 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:13.982749    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-5vdc7\" (UID: \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"
Jul 07 15:06:13.983073 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:13.982872    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-istio-token\") pod \"istiod-openshift-gateway-7574459656-5vdc7\" (UID: \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"
Jul 07 15:06:14.083220 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:14.083179    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k9gz\" (UniqueName: \"kubernetes.io/projected/f8aeca71-6370-4687-8796-6bbe61f3bc92-kube-api-access-8k9gz\") pod \"model-serving-api-54d89bd5fc-5f6g2\" (UID: \"f8aeca71-6370-4687-8796-6bbe61f3bc92\") " pod="opendatahub/model-serving-api-54d89bd5fc-5f6g2"
Jul 07 15:06:14.083220 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:14.083222    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-local-certs\") pod \"istiod-openshift-gateway-7574459656-5vdc7\" (UID: \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"
Jul 07 15:06:14.083489 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:14.083250    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v65r4\" (UniqueName: \"kubernetes.io/projected/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-kube-api-access-v65r4\") pod \"istiod-openshift-gateway-7574459656-5vdc7\" (UID: \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"
Jul 07 15:06:14.083489 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:14.083280    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8aeca71-6370-4687-8796-6bbe61f3bc92-tls-certs\") pod \"model-serving-api-54d89bd5fc-5f6g2\" (UID: \"f8aeca71-6370-4687-8796-6bbe61f3bc92\") " pod="opendatahub/model-serving-api-54d89bd5fc-5f6g2"
Jul 07 15:06:14.083489 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:14.083300    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-cacerts\") pod \"istiod-openshift-gateway-7574459656-5vdc7\" (UID: \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"
Jul 07 15:06:14.083489 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:14.083326    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-5vdc7\" (UID: \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"
Jul 07 15:06:14.083489 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:14.083354    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-5vdc7\" (UID: \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"
Jul 07 15:06:14.083489 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:14.083383    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-5vdc7\" (UID: \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"
Jul 07 15:06:14.083489 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:14.083468    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-istio-token\") pod \"istiod-openshift-gateway-7574459656-5vdc7\" (UID: \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"
Jul 07 15:06:14.084355 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:14.084327    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-5vdc7\" (UID: \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"
Jul 07 15:06:14.085920 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:14.085891    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-5vdc7\" (UID: \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"
Jul 07 15:06:14.086033 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:14.085983    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-cacerts\") pod \"istiod-openshift-gateway-7574459656-5vdc7\" (UID: \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"
Jul 07 15:06:14.086084 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:14.086027    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8aeca71-6370-4687-8796-6bbe61f3bc92-tls-certs\") pod \"model-serving-api-54d89bd5fc-5f6g2\" (UID: \"f8aeca71-6370-4687-8796-6bbe61f3bc92\") " pod="opendatahub/model-serving-api-54d89bd5fc-5f6g2"
Jul 07 15:06:14.086207 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:14.086185    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-local-certs\") pod \"istiod-openshift-gateway-7574459656-5vdc7\" (UID: \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"
Jul 07 15:06:14.086309 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:14.086294    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-5vdc7\" (UID: \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"
Jul 07 15:06:14.097966 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:14.097935    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-istio-token\") pod \"istiod-openshift-gateway-7574459656-5vdc7\" (UID: \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"
Jul 07 15:06:14.097966 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:14.097947    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k9gz\" (UniqueName: \"kubernetes.io/projected/f8aeca71-6370-4687-8796-6bbe61f3bc92-kube-api-access-8k9gz\") pod \"model-serving-api-54d89bd5fc-5f6g2\" (UID: \"f8aeca71-6370-4687-8796-6bbe61f3bc92\") " pod="opendatahub/model-serving-api-54d89bd5fc-5f6g2"
Jul 07 15:06:14.098118 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:14.097953    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v65r4\" (UniqueName: \"kubernetes.io/projected/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-kube-api-access-v65r4\") pod \"istiod-openshift-gateway-7574459656-5vdc7\" (UID: \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"
Jul 07 15:06:14.181452 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:14.181419    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7r9lh"]
Jul 07 15:06:14.183825 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:06:14.183799    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17173b3e_ac3b_4761_892d_be6bb3a484c3.slice/crio-f6329905801a2d31754c0a4ffd04657ff7fe24b3d44686c392084d61343c2738 WatchSource:0}: Error finding container f6329905801a2d31754c0a4ffd04657ff7fe24b3d44686c392084d61343c2738: Status 404 returned error can't find the container with id f6329905801a2d31754c0a4ffd04657ff7fe24b3d44686c392084d61343c2738
Jul 07 15:06:14.203983 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:14.203954    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-5f6g2"
Jul 07 15:06:14.238416 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:14.238385    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"
Jul 07 15:06:14.352069 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:14.352026    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-7r9lh" event={"ID":"17173b3e-ac3b-4761-892d-be6bb3a484c3","Type":"ContainerStarted","Data":"f6329905801a2d31754c0a4ffd04657ff7fe24b3d44686c392084d61343c2738"}
Jul 07 15:06:14.374613 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:14.374583    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5f6g2"]
Jul 07 15:06:14.377636 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:06:14.377593    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8aeca71_6370_4687_8796_6bbe61f3bc92.slice/crio-8ff575fd6c12afa7ebe6b74713c14b1e95cab2fff575d7e8417fef6428f14dd7 WatchSource:0}: Error finding container 8ff575fd6c12afa7ebe6b74713c14b1e95cab2fff575d7e8417fef6428f14dd7: Status 404 returned error can't find the container with id 8ff575fd6c12afa7ebe6b74713c14b1e95cab2fff575d7e8417fef6428f14dd7
Jul 07 15:06:14.439656 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:14.439625    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"]
Jul 07 15:06:14.440229 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:06:14.440193    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4ca0cd8_d8a8_48b9_a584_bf5c5fb7e9f2.slice/crio-40f57bdd4e3b28618ffdd60122b863b5d74d8ebe5be660e073903bbddca17c48 WatchSource:0}: Error finding container 40f57bdd4e3b28618ffdd60122b863b5d74d8ebe5be660e073903bbddca17c48: Status 404 returned error can't find the container with id 40f57bdd4e3b28618ffdd60122b863b5d74d8ebe5be660e073903bbddca17c48
Jul 07 15:06:15.363975 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:15.363806    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7" event={"ID":"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2","Type":"ContainerStarted","Data":"40f57bdd4e3b28618ffdd60122b863b5d74d8ebe5be660e073903bbddca17c48"}
Jul 07 15:06:15.366921 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:15.366812    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-5f6g2" event={"ID":"f8aeca71-6370-4687-8796-6bbe61f3bc92","Type":"ContainerStarted","Data":"8ff575fd6c12afa7ebe6b74713c14b1e95cab2fff575d7e8417fef6428f14dd7"}
Jul 07 15:06:19.465968 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:19.465937    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:06:19.466351 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:19.466008    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:06:20.393539 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:20.393486    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8697c578dd-klxhp" event={"ID":"c2338f6c-01a4-4b70-81c8-79bb3caecad7","Type":"ContainerStarted","Data":"a89ad9a9894ff13e609c4bdc4a94aa47d706fc8095451b46c714b1ab352fd332"}
Jul 07 15:06:20.395255 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:20.395229    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-7r9lh" event={"ID":"17173b3e-ac3b-4761-892d-be6bb3a484c3","Type":"ContainerStarted","Data":"99d4221ee339090ec45e4ee7b3856a8d507c140bf9ed3c4f555280f93975f56b"}
Jul 07 15:06:20.395389 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:20.395356    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-7r9lh"
Jul 07 15:06:20.396942 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:20.396917    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7" event={"ID":"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2","Type":"ContainerStarted","Data":"7e97cc4fcc60204ae67c72788af22a383effd0926ce996a632f0a4265ea967e0"}
Jul 07 15:06:20.397041 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:20.396966    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"
Jul 07 15:06:20.399049 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:20.399026    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-5f6g2" event={"ID":"f8aeca71-6370-4687-8796-6bbe61f3bc92","Type":"ContainerStarted","Data":"6b5d6ab79ca49ec152696bc5f0864eccbd58bb1930e458eef36a82d5a79160e5"}
Jul 07 15:06:20.399614 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:20.399595    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-5f6g2"
Jul 07 15:06:20.399993 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:20.399721    2567 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-5vdc7 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 07 15:06:20.399993 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:20.399770    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7" podUID="b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:06:20.436073 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:20.435563    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-8697c578dd-klxhp" podStartSLOduration=1.892028554 podStartE2EDuration="8.435540487s" totalImagesPullingTime="6.543511933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:06:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:06:20.433812304 +0000 UTC m=+348.230349053" watchObservedRunningTime="2026-07-07 15:06:20.435540487 +0000 UTC m=+348.232077216"
Jul 07 15:06:20.512613 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:20.512331    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-7r9lh" podStartSLOduration=3.188145842 podStartE2EDuration="8.512306344s" totalImagesPullingTime="5.324160502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:06:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:06:20.511239301 +0000 UTC m=+348.307776030" watchObservedRunningTime="2026-07-07 15:06:20.512306344 +0000 UTC m=+348.308843073"
Jul 07 15:06:20.551382 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:20.551332    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-5f6g2" podStartSLOduration=2.468797764 podStartE2EDuration="7.55131592s" totalImagesPullingTime="5.082518156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:06:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:06:20.550157705 +0000 UTC m=+348.346694453" watchObservedRunningTime="2026-07-07 15:06:20.55131592 +0000 UTC m=+348.347852649"
Jul 07 15:06:20.593134 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:20.593081    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7" podStartSLOduration=2.569679496 podStartE2EDuration="7.593061931s" totalImagesPullingTime="5.023382435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:06:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:06:20.59091264 +0000 UTC m=+348.387449388" watchObservedRunningTime="2026-07-07 15:06:20.593061931 +0000 UTC m=+348.389598659"
Jul 07 15:06:21.407198 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:21.407161    2567 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-5vdc7 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 07 15:06:21.407416 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:21.407240    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7" podUID="b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:06:24.407093 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:24.407016    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"
Jul 07 15:06:26.973733 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:26.973697    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-dgck8"]
Jul 07 15:06:26.980675 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:26.980648    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dgck8"
Jul 07 15:06:26.985703 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:26.985306    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-9ftc8\""
Jul 07 15:06:26.985703 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:26.985560    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 07 15:06:27.000199 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:27.000168    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-dgck8"]
Jul 07 15:06:27.001860 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:27.001814    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gksh\" (UniqueName: \"kubernetes.io/projected/023e88fa-bc7c-4a44-be91-311ef63c3fda-kube-api-access-5gksh\") pod \"llmisvc-controller-manager-777c6bd7d5-dgck8\" (UID: \"023e88fa-bc7c-4a44-be91-311ef63c3fda\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dgck8"
Jul 07 15:06:27.002018 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:27.001994    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/023e88fa-bc7c-4a44-be91-311ef63c3fda-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dgck8\" (UID: \"023e88fa-bc7c-4a44-be91-311ef63c3fda\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dgck8"
Jul 07 15:06:27.103351 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:27.103309    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/023e88fa-bc7c-4a44-be91-311ef63c3fda-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dgck8\" (UID: \"023e88fa-bc7c-4a44-be91-311ef63c3fda\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dgck8"
Jul 07 15:06:27.103586 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:27.103441    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gksh\" (UniqueName: \"kubernetes.io/projected/023e88fa-bc7c-4a44-be91-311ef63c3fda-kube-api-access-5gksh\") pod \"llmisvc-controller-manager-777c6bd7d5-dgck8\" (UID: \"023e88fa-bc7c-4a44-be91-311ef63c3fda\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dgck8"
Jul 07 15:06:27.103586 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:06:27.103462    2567 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 07 15:06:27.103586 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:06:27.103548    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/023e88fa-bc7c-4a44-be91-311ef63c3fda-cert podName:023e88fa-bc7c-4a44-be91-311ef63c3fda nodeName:}" failed. No retries permitted until 2026-07-07 15:06:27.603527204 +0000 UTC m=+355.400063913 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/023e88fa-bc7c-4a44-be91-311ef63c3fda-cert") pod "llmisvc-controller-manager-777c6bd7d5-dgck8" (UID: "023e88fa-bc7c-4a44-be91-311ef63c3fda") : secret "llmisvc-webhook-server-cert" not found
Jul 07 15:06:27.117000 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:27.116969    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gksh\" (UniqueName: \"kubernetes.io/projected/023e88fa-bc7c-4a44-be91-311ef63c3fda-kube-api-access-5gksh\") pod \"llmisvc-controller-manager-777c6bd7d5-dgck8\" (UID: \"023e88fa-bc7c-4a44-be91-311ef63c3fda\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dgck8"
Jul 07 15:06:27.607892 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:27.607818    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/023e88fa-bc7c-4a44-be91-311ef63c3fda-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dgck8\" (UID: \"023e88fa-bc7c-4a44-be91-311ef63c3fda\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dgck8"
Jul 07 15:06:27.610485 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:27.610457    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/023e88fa-bc7c-4a44-be91-311ef63c3fda-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dgck8\" (UID: \"023e88fa-bc7c-4a44-be91-311ef63c3fda\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dgck8"
Jul 07 15:06:27.896722 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:27.896638    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dgck8"
Jul 07 15:06:28.050249 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:28.050225    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-dgck8"]
Jul 07 15:06:28.052771 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:06:28.052740    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod023e88fa_bc7c_4a44_be91_311ef63c3fda.slice/crio-d96c751af34a296ec6a3d5acd50debe8425fe003a25b47ec735bc29be9ecc4fe WatchSource:0}: Error finding container d96c751af34a296ec6a3d5acd50debe8425fe003a25b47ec735bc29be9ecc4fe: Status 404 returned error can't find the container with id d96c751af34a296ec6a3d5acd50debe8425fe003a25b47ec735bc29be9ecc4fe
Jul 07 15:06:28.435903 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:28.435867    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dgck8" event={"ID":"023e88fa-bc7c-4a44-be91-311ef63c3fda","Type":"ContainerStarted","Data":"d96c751af34a296ec6a3d5acd50debe8425fe003a25b47ec735bc29be9ecc4fe"}
Jul 07 15:06:28.745575 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:28.745477    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"]
Jul 07 15:06:28.745748 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:28.745681    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7" podUID="b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2" containerName="discovery" containerID="cri-o://7e97cc4fcc60204ae67c72788af22a383effd0926ce996a632f0a4265ea967e0" gracePeriod=30
Jul 07 15:06:29.004107 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.004050    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"
Jul 07 15:06:29.120086 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.120050    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v65r4\" (UniqueName: \"kubernetes.io/projected/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-kube-api-access-v65r4\") pod \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\" (UID: \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\") "
Jul 07 15:06:29.120549 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.120108    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-istio-token\") pod \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\" (UID: \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\") "
Jul 07 15:06:29.120549 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.120141    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-cacerts\") pod \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\" (UID: \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\") "
Jul 07 15:06:29.120549 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.120168    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-istio-kubeconfig\") pod \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\" (UID: \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\") "
Jul 07 15:06:29.120549 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.120188    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-istio-csr-dns-cert\") pod \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\" (UID: \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\") "
Jul 07 15:06:29.120549 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.120243    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-istio-csr-ca-configmap\") pod \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\" (UID: \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\") "
Jul 07 15:06:29.120549 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.120324    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-local-certs\") pod \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\" (UID: \"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2\") "
Jul 07 15:06:29.121105 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.120775    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2" (UID: "b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:06:29.122959 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.122915    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-cacerts" (OuterVolumeSpecName: "cacerts") pod "b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2" (UID: "b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:06:29.122959 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.122931    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-local-certs" (OuterVolumeSpecName: "local-certs") pod "b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2" (UID: "b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:06:29.123129 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.122951    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2" (UID: "b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:06:29.123129 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.122973    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-istio-token" (OuterVolumeSpecName: "istio-token") pod "b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2" (UID: "b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:06:29.123129 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.123058    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2" (UID: "b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:06:29.123129 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.123097    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-kube-api-access-v65r4" (OuterVolumeSpecName: "kube-api-access-v65r4") pod "b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2" (UID: "b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2"). InnerVolumeSpecName "kube-api-access-v65r4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:06:29.221360 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.221325    2567 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-local-certs\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:06:29.221360 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.221354    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v65r4\" (UniqueName: \"kubernetes.io/projected/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-kube-api-access-v65r4\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:06:29.221360 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.221365    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-istio-token\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:06:29.221360 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.221374    2567 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-cacerts\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:06:29.221682 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.221382    2567 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-istio-kubeconfig\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:06:29.221682 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.221389    2567 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-istio-csr-dns-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:06:29.221682 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.221397    2567 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2-istio-csr-ca-configmap\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:06:29.442057 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.442020    2567 generic.go:356] "Generic (PLEG): container finished" podID="b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2" containerID="7e97cc4fcc60204ae67c72788af22a383effd0926ce996a632f0a4265ea967e0" exitCode=0
Jul 07 15:06:29.442252 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.442080    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"
Jul 07 15:06:29.442252 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.442091    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7" event={"ID":"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2","Type":"ContainerDied","Data":"7e97cc4fcc60204ae67c72788af22a383effd0926ce996a632f0a4265ea967e0"}
Jul 07 15:06:29.442252 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.442173    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7" event={"ID":"b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2","Type":"ContainerDied","Data":"40f57bdd4e3b28618ffdd60122b863b5d74d8ebe5be660e073903bbddca17c48"}
Jul 07 15:06:29.442252 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.442204    2567 scope.go:117] "RemoveContainer" containerID="7e97cc4fcc60204ae67c72788af22a383effd0926ce996a632f0a4265ea967e0"
Jul 07 15:06:29.473023 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.472992    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"]
Jul 07 15:06:29.494610 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.494575    2567 scope.go:117] "RemoveContainer" containerID="7e97cc4fcc60204ae67c72788af22a383effd0926ce996a632f0a4265ea967e0"
Jul 07 15:06:29.494977 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:06:29.494955    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e97cc4fcc60204ae67c72788af22a383effd0926ce996a632f0a4265ea967e0\": container with ID starting with 7e97cc4fcc60204ae67c72788af22a383effd0926ce996a632f0a4265ea967e0 not found: ID does not exist" containerID="7e97cc4fcc60204ae67c72788af22a383effd0926ce996a632f0a4265ea967e0"
Jul 07 15:06:29.495065 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.494987    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e97cc4fcc60204ae67c72788af22a383effd0926ce996a632f0a4265ea967e0"} err="failed to get container status \"7e97cc4fcc60204ae67c72788af22a383effd0926ce996a632f0a4265ea967e0\": rpc error: code = NotFound desc = could not find container \"7e97cc4fcc60204ae67c72788af22a383effd0926ce996a632f0a4265ea967e0\": container with ID starting with 7e97cc4fcc60204ae67c72788af22a383effd0926ce996a632f0a4265ea967e0 not found: ID does not exist"
Jul 07 15:06:29.498270 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:29.498243    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5vdc7"]
Jul 07 15:06:30.777319 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:30.777284    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2" path="/var/lib/kubelet/pods/b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2/volumes"
Jul 07 15:06:31.407764 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:31.407725    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-7r9lh"
Jul 07 15:06:32.414595 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:32.414560    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-5f6g2"
Jul 07 15:06:32.480488 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:32.480455    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dgck8" event={"ID":"023e88fa-bc7c-4a44-be91-311ef63c3fda","Type":"ContainerStarted","Data":"28b331df5b3ca3f4c5f25b01d67e19c3808df4d53a3c988b4eec98d1a41d317d"}
Jul 07 15:06:32.480669 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:32.480587    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dgck8"
Jul 07 15:06:32.539328 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:06:32.539274    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dgck8" podStartSLOduration=2.66301475 podStartE2EDuration="6.539256716s" totalImagesPullingTime="3.876241966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:06:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:06:32.536786252 +0000 UTC m=+360.333322980" watchObservedRunningTime="2026-07-07 15:06:32.539256716 +0000 UTC m=+360.335793443"
Jul 07 15:07:03.487436 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:03.487406    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dgck8"
Jul 07 15:07:13.805625 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:13.805592    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dnb8n"]
Jul 07 15:07:13.806032 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:13.805961    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2" containerName="discovery"
Jul 07 15:07:13.806032 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:13.805973    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2" containerName="discovery"
Jul 07 15:07:13.806032 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:13.806023    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4ca0cd8-d8a8-48b9-a584-bf5c5fb7e9f2" containerName="discovery"
Jul 07 15:07:13.807878 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:13.807860    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dnb8n"
Jul 07 15:07:13.811032 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:13.811010    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 07 15:07:13.811195 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:13.811045    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-z768d\""
Jul 07 15:07:13.812230 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:13.812212    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 07 15:07:13.823546 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:13.823512    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dnb8n"]
Jul 07 15:07:13.898323 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:13.898288    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9skx\" (UniqueName: \"kubernetes.io/projected/dc626485-7dd2-4929-b920-abd0d2a454ac-kube-api-access-s9skx\") pod \"kuadrant-operator-catalog-dnb8n\" (UID: \"dc626485-7dd2-4929-b920-abd0d2a454ac\") " pod="kuadrant-system/kuadrant-operator-catalog-dnb8n"
Jul 07 15:07:13.999340 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:13.999292    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9skx\" (UniqueName: \"kubernetes.io/projected/dc626485-7dd2-4929-b920-abd0d2a454ac-kube-api-access-s9skx\") pod \"kuadrant-operator-catalog-dnb8n\" (UID: \"dc626485-7dd2-4929-b920-abd0d2a454ac\") " pod="kuadrant-system/kuadrant-operator-catalog-dnb8n"
Jul 07 15:07:14.015066 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:14.015030    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9skx\" (UniqueName: \"kubernetes.io/projected/dc626485-7dd2-4929-b920-abd0d2a454ac-kube-api-access-s9skx\") pod \"kuadrant-operator-catalog-dnb8n\" (UID: \"dc626485-7dd2-4929-b920-abd0d2a454ac\") " pod="kuadrant-system/kuadrant-operator-catalog-dnb8n"
Jul 07 15:07:14.118752 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:14.118664    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dnb8n"
Jul 07 15:07:14.167124 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:14.167089    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dnb8n"]
Jul 07 15:07:14.285464 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:14.285378    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dnb8n"]
Jul 07 15:07:14.288124 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:07:14.288098    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc626485_7dd2_4929_b920_abd0d2a454ac.slice/crio-fd8de5fee2caf8cc7b815f4c526a32707a713497a73f7bc92bdee44589ca409c WatchSource:0}: Error finding container fd8de5fee2caf8cc7b815f4c526a32707a713497a73f7bc92bdee44589ca409c: Status 404 returned error can't find the container with id fd8de5fee2caf8cc7b815f4c526a32707a713497a73f7bc92bdee44589ca409c
Jul 07 15:07:14.370252 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:14.370164    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lct58"]
Jul 07 15:07:14.372990 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:14.372972    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lct58"
Jul 07 15:07:14.390061 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:14.390033    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lct58"]
Jul 07 15:07:14.402749 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:14.402716    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzwmm\" (UniqueName: \"kubernetes.io/projected/7519d06b-9629-49d3-a92e-9d267fa5ac0d-kube-api-access-qzwmm\") pod \"kuadrant-operator-catalog-lct58\" (UID: \"7519d06b-9629-49d3-a92e-9d267fa5ac0d\") " pod="kuadrant-system/kuadrant-operator-catalog-lct58"
Jul 07 15:07:14.503974 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:14.503937    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzwmm\" (UniqueName: \"kubernetes.io/projected/7519d06b-9629-49d3-a92e-9d267fa5ac0d-kube-api-access-qzwmm\") pod \"kuadrant-operator-catalog-lct58\" (UID: \"7519d06b-9629-49d3-a92e-9d267fa5ac0d\") " pod="kuadrant-system/kuadrant-operator-catalog-lct58"
Jul 07 15:07:14.518588 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:14.518562    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzwmm\" (UniqueName: \"kubernetes.io/projected/7519d06b-9629-49d3-a92e-9d267fa5ac0d-kube-api-access-qzwmm\") pod \"kuadrant-operator-catalog-lct58\" (UID: \"7519d06b-9629-49d3-a92e-9d267fa5ac0d\") " pod="kuadrant-system/kuadrant-operator-catalog-lct58"
Jul 07 15:07:14.664150 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:14.664059    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dnb8n" event={"ID":"dc626485-7dd2-4929-b920-abd0d2a454ac","Type":"ContainerStarted","Data":"fd8de5fee2caf8cc7b815f4c526a32707a713497a73f7bc92bdee44589ca409c"}
Jul 07 15:07:14.682335 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:14.682305    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lct58"
Jul 07 15:07:14.838208 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:14.838184    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lct58"]
Jul 07 15:07:14.873903 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:07:14.873856    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7519d06b_9629_49d3_a92e_9d267fa5ac0d.slice/crio-6e0e1d9e49a8344deb24a33947c147f3a2076f54b6fa70490f7ac3f82fadb962 WatchSource:0}: Error finding container 6e0e1d9e49a8344deb24a33947c147f3a2076f54b6fa70490f7ac3f82fadb962: Status 404 returned error can't find the container with id 6e0e1d9e49a8344deb24a33947c147f3a2076f54b6fa70490f7ac3f82fadb962
Jul 07 15:07:15.669114 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:15.669080    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lct58" event={"ID":"7519d06b-9629-49d3-a92e-9d267fa5ac0d","Type":"ContainerStarted","Data":"6e0e1d9e49a8344deb24a33947c147f3a2076f54b6fa70490f7ac3f82fadb962"}
Jul 07 15:07:16.676254 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:16.676218    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lct58" event={"ID":"7519d06b-9629-49d3-a92e-9d267fa5ac0d","Type":"ContainerStarted","Data":"51e2c3611f46f6c91550426ea637612d3eda1386907bfa02e1be53c3d623b963"}
Jul 07 15:07:16.677852 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:16.677811    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dnb8n" event={"ID":"dc626485-7dd2-4929-b920-abd0d2a454ac","Type":"ContainerStarted","Data":"54733afa93969b622790b41cc8eeb004f3ec9a48b795d6e6e677563032e860fe"}
Jul 07 15:07:16.677964 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:16.677913    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-dnb8n" podUID="dc626485-7dd2-4929-b920-abd0d2a454ac" containerName="registry-server" containerID="cri-o://54733afa93969b622790b41cc8eeb004f3ec9a48b795d6e6e677563032e860fe" gracePeriod=2
Jul 07 15:07:16.698990 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:16.698936    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-lct58" podStartSLOduration=1.116305343 podStartE2EDuration="2.698917963s" totalImagesPullingTime="1.58261262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:07:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:07:16.698150252 +0000 UTC m=+404.494686980" watchObservedRunningTime="2026-07-07 15:07:16.698917963 +0000 UTC m=+404.495454668"
Jul 07 15:07:16.721985 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:16.721879    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-dnb8n" podStartSLOduration=1.556230868 podStartE2EDuration="3.721865175s" totalImagesPullingTime="2.165634307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:07:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:07:16.720103818 +0000 UTC m=+404.516640537" watchObservedRunningTime="2026-07-07 15:07:16.721865175 +0000 UTC m=+404.518401902"
Jul 07 15:07:16.920528 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:16.920498    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dnb8n"
Jul 07 15:07:17.023683 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:17.023587    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9skx\" (UniqueName: \"kubernetes.io/projected/dc626485-7dd2-4929-b920-abd0d2a454ac-kube-api-access-s9skx\") pod \"dc626485-7dd2-4929-b920-abd0d2a454ac\" (UID: \"dc626485-7dd2-4929-b920-abd0d2a454ac\") "
Jul 07 15:07:17.025961 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:17.025939    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc626485-7dd2-4929-b920-abd0d2a454ac-kube-api-access-s9skx" (OuterVolumeSpecName: "kube-api-access-s9skx") pod "dc626485-7dd2-4929-b920-abd0d2a454ac" (UID: "dc626485-7dd2-4929-b920-abd0d2a454ac"). InnerVolumeSpecName "kube-api-access-s9skx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:07:17.124878 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:17.124816    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9skx\" (UniqueName: \"kubernetes.io/projected/dc626485-7dd2-4929-b920-abd0d2a454ac-kube-api-access-s9skx\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:07:17.682987 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:17.682954    2567 generic.go:356] "Generic (PLEG): container finished" podID="dc626485-7dd2-4929-b920-abd0d2a454ac" containerID="54733afa93969b622790b41cc8eeb004f3ec9a48b795d6e6e677563032e860fe" exitCode=0
Jul 07 15:07:17.683378 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:17.682995    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dnb8n"
Jul 07 15:07:17.683378 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:17.683034    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dnb8n" event={"ID":"dc626485-7dd2-4929-b920-abd0d2a454ac","Type":"ContainerDied","Data":"54733afa93969b622790b41cc8eeb004f3ec9a48b795d6e6e677563032e860fe"}
Jul 07 15:07:17.683378 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:17.683084    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dnb8n" event={"ID":"dc626485-7dd2-4929-b920-abd0d2a454ac","Type":"ContainerDied","Data":"fd8de5fee2caf8cc7b815f4c526a32707a713497a73f7bc92bdee44589ca409c"}
Jul 07 15:07:17.683378 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:17.683109    2567 scope.go:117] "RemoveContainer" containerID="54733afa93969b622790b41cc8eeb004f3ec9a48b795d6e6e677563032e860fe"
Jul 07 15:07:17.692571 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:17.692553    2567 scope.go:117] "RemoveContainer" containerID="54733afa93969b622790b41cc8eeb004f3ec9a48b795d6e6e677563032e860fe"
Jul 07 15:07:17.692862 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:07:17.692817    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54733afa93969b622790b41cc8eeb004f3ec9a48b795d6e6e677563032e860fe\": container with ID starting with 54733afa93969b622790b41cc8eeb004f3ec9a48b795d6e6e677563032e860fe not found: ID does not exist" containerID="54733afa93969b622790b41cc8eeb004f3ec9a48b795d6e6e677563032e860fe"
Jul 07 15:07:17.692916 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:17.692874    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54733afa93969b622790b41cc8eeb004f3ec9a48b795d6e6e677563032e860fe"} err="failed to get container status \"54733afa93969b622790b41cc8eeb004f3ec9a48b795d6e6e677563032e860fe\": rpc error: code = NotFound desc = could not find container \"54733afa93969b622790b41cc8eeb004f3ec9a48b795d6e6e677563032e860fe\": container with ID starting with 54733afa93969b622790b41cc8eeb004f3ec9a48b795d6e6e677563032e860fe not found: ID does not exist"
Jul 07 15:07:17.717141 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:17.717104    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dnb8n"]
Jul 07 15:07:17.719367 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:17.719342    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dnb8n"]
Jul 07 15:07:18.777808 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:18.777757    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc626485-7dd2-4929-b920-abd0d2a454ac" path="/var/lib/kubelet/pods/dc626485-7dd2-4929-b920-abd0d2a454ac/volumes"
Jul 07 15:07:24.682644 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:24.682609    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-lct58"
Jul 07 15:07:24.683089 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:24.682674    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-lct58"
Jul 07 15:07:24.705643 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:24.705617    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-lct58"
Jul 07 15:07:24.735782 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:24.735755    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-lct58"
Jul 07 15:07:43.096606 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:43.096571    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-hj44h"]
Jul 07 15:07:43.097085 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:43.096927    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc626485-7dd2-4929-b920-abd0d2a454ac" containerName="registry-server"
Jul 07 15:07:43.097085 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:43.096939    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc626485-7dd2-4929-b920-abd0d2a454ac" containerName="registry-server"
Jul 07 15:07:43.097085 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:43.097010    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc626485-7dd2-4929-b920-abd0d2a454ac" containerName="registry-server"
Jul 07 15:07:43.099483 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:43.099465    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hj44h"
Jul 07 15:07:43.103357 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:43.103333    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-sxv2m\""
Jul 07 15:07:43.114163 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:43.114133    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-hj44h"]
Jul 07 15:07:43.247981 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:43.247943    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsrcp\" (UniqueName: \"kubernetes.io/projected/05c688f6-180a-4a2c-a7c1-0b3ad743666b-kube-api-access-lsrcp\") pod \"authorino-operator-75cc8d5cb7-hj44h\" (UID: \"05c688f6-180a-4a2c-a7c1-0b3ad743666b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-hj44h"
Jul 07 15:07:43.349110 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:43.349015    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsrcp\" (UniqueName: \"kubernetes.io/projected/05c688f6-180a-4a2c-a7c1-0b3ad743666b-kube-api-access-lsrcp\") pod \"authorino-operator-75cc8d5cb7-hj44h\" (UID: \"05c688f6-180a-4a2c-a7c1-0b3ad743666b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-hj44h"
Jul 07 15:07:43.364353 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:43.364324    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsrcp\" (UniqueName: \"kubernetes.io/projected/05c688f6-180a-4a2c-a7c1-0b3ad743666b-kube-api-access-lsrcp\") pod \"authorino-operator-75cc8d5cb7-hj44h\" (UID: \"05c688f6-180a-4a2c-a7c1-0b3ad743666b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-hj44h"
Jul 07 15:07:43.409299 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:43.409270    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hj44h"
Jul 07 15:07:43.549183 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:43.549161    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-hj44h"]
Jul 07 15:07:43.551422 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:07:43.551392    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05c688f6_180a_4a2c_a7c1_0b3ad743666b.slice/crio-1d3475a2bc1203f9359c4c7fc5a87ba48d3983cdc2beb6853339f3940f230041 WatchSource:0}: Error finding container 1d3475a2bc1203f9359c4c7fc5a87ba48d3983cdc2beb6853339f3940f230041: Status 404 returned error can't find the container with id 1d3475a2bc1203f9359c4c7fc5a87ba48d3983cdc2beb6853339f3940f230041
Jul 07 15:07:43.796198 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:43.796107    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hj44h" event={"ID":"05c688f6-180a-4a2c-a7c1-0b3ad743666b","Type":"ContainerStarted","Data":"1d3475a2bc1203f9359c4c7fc5a87ba48d3983cdc2beb6853339f3940f230041"}
Jul 07 15:07:46.323504 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:46.323410    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k4dqc"]
Jul 07 15:07:46.326848 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:46.326816    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k4dqc"
Jul 07 15:07:46.330317 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:46.330294    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-92nzr\""
Jul 07 15:07:46.339747 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:46.339725    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k4dqc"]
Jul 07 15:07:46.478195 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:46.478160    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwspp\" (UniqueName: \"kubernetes.io/projected/f36e272b-dd1e-4d7f-b50b-a0f89bd69412-kube-api-access-lwspp\") pod \"limitador-operator-controller-manager-859d7b78bd-k4dqc\" (UID: \"f36e272b-dd1e-4d7f-b50b-a0f89bd69412\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k4dqc"
Jul 07 15:07:46.579271 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:46.579173    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwspp\" (UniqueName: \"kubernetes.io/projected/f36e272b-dd1e-4d7f-b50b-a0f89bd69412-kube-api-access-lwspp\") pod \"limitador-operator-controller-manager-859d7b78bd-k4dqc\" (UID: \"f36e272b-dd1e-4d7f-b50b-a0f89bd69412\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k4dqc"
Jul 07 15:07:46.589980 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:46.589953    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwspp\" (UniqueName: \"kubernetes.io/projected/f36e272b-dd1e-4d7f-b50b-a0f89bd69412-kube-api-access-lwspp\") pod \"limitador-operator-controller-manager-859d7b78bd-k4dqc\" (UID: \"f36e272b-dd1e-4d7f-b50b-a0f89bd69412\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k4dqc"
Jul 07 15:07:46.637055 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:46.637004    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k4dqc"
Jul 07 15:07:46.780189 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:46.780094    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k4dqc"]
Jul 07 15:07:46.782626 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:07:46.782600    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf36e272b_dd1e_4d7f_b50b_a0f89bd69412.slice/crio-bb4992d07d99fef401749762c1e9fad53458ff8a7d0836abded90342bd882a65 WatchSource:0}: Error finding container bb4992d07d99fef401749762c1e9fad53458ff8a7d0836abded90342bd882a65: Status 404 returned error can't find the container with id bb4992d07d99fef401749762c1e9fad53458ff8a7d0836abded90342bd882a65
Jul 07 15:07:46.811947 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:46.811913    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hj44h" event={"ID":"05c688f6-180a-4a2c-a7c1-0b3ad743666b","Type":"ContainerStarted","Data":"9cc965dda9c81848fd1a13815f203973e2e134df9a09fb6503bcb48b672f2511"}
Jul 07 15:07:46.812118 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:46.812033    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hj44h"
Jul 07 15:07:46.813638 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:46.813564    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k4dqc" event={"ID":"f36e272b-dd1e-4d7f-b50b-a0f89bd69412","Type":"ContainerStarted","Data":"bb4992d07d99fef401749762c1e9fad53458ff8a7d0836abded90342bd882a65"}
Jul 07 15:07:46.833796 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:46.833689    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hj44h" podStartSLOduration=1.336124723 podStartE2EDuration="3.83367225s" totalImagesPullingTime="2.497547527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:07:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:07:46.833531511 +0000 UTC m=+434.630068261" watchObservedRunningTime="2026-07-07 15:07:46.83367225 +0000 UTC m=+434.630208972"
Jul 07 15:07:48.973055 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:48.973020    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-t6tmh"]
Jul 07 15:07:48.976508 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:48.976484    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t6tmh"
Jul 07 15:07:48.979628 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:48.979607    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 07 15:07:48.979752 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:48.979686    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-5ct9m\""
Jul 07 15:07:48.991682 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:48.991655    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-t6tmh"]
Jul 07 15:07:49.102166 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:49.102128    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p55n\" (UniqueName: \"kubernetes.io/projected/539a21f4-9e00-4fa5-bff8-2f5388906b1b-kube-api-access-5p55n\") pod \"dns-operator-controller-manager-864dd78c4c-t6tmh\" (UID: \"539a21f4-9e00-4fa5-bff8-2f5388906b1b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t6tmh"
Jul 07 15:07:49.202607 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:49.202566    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p55n\" (UniqueName: \"kubernetes.io/projected/539a21f4-9e00-4fa5-bff8-2f5388906b1b-kube-api-access-5p55n\") pod \"dns-operator-controller-manager-864dd78c4c-t6tmh\" (UID: \"539a21f4-9e00-4fa5-bff8-2f5388906b1b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t6tmh"
Jul 07 15:07:49.213746 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:49.213716    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p55n\" (UniqueName: \"kubernetes.io/projected/539a21f4-9e00-4fa5-bff8-2f5388906b1b-kube-api-access-5p55n\") pod \"dns-operator-controller-manager-864dd78c4c-t6tmh\" (UID: \"539a21f4-9e00-4fa5-bff8-2f5388906b1b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t6tmh"
Jul 07 15:07:49.286594 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:49.286555    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t6tmh"
Jul 07 15:07:49.444026 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:49.443990    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-t6tmh"]
Jul 07 15:07:49.447507 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:07:49.447483    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod539a21f4_9e00_4fa5_bff8_2f5388906b1b.slice/crio-608c2e4f8acc276119acd69a0800834adb64d9a554337df001ceb0ab5f3d3cdb WatchSource:0}: Error finding container 608c2e4f8acc276119acd69a0800834adb64d9a554337df001ceb0ab5f3d3cdb: Status 404 returned error can't find the container with id 608c2e4f8acc276119acd69a0800834adb64d9a554337df001ceb0ab5f3d3cdb
Jul 07 15:07:49.829558 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:49.829516    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t6tmh" event={"ID":"539a21f4-9e00-4fa5-bff8-2f5388906b1b","Type":"ContainerStarted","Data":"608c2e4f8acc276119acd69a0800834adb64d9a554337df001ceb0ab5f3d3cdb"}
Jul 07 15:07:49.830885 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:49.830861    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k4dqc" event={"ID":"f36e272b-dd1e-4d7f-b50b-a0f89bd69412","Type":"ContainerStarted","Data":"accb8bc8a5d9baeecb9fc9d56bfae5222c0b4f0738158fd43f32dc3f00bb70b9"}
Jul 07 15:07:49.831000 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:49.830986    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k4dqc"
Jul 07 15:07:49.857646 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:49.857587    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k4dqc" podStartSLOduration=1.847723931 podStartE2EDuration="3.857570159s" totalImagesPullingTime="2.009846228s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:07:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:07:49.855773193 +0000 UTC m=+437.652309922" watchObservedRunningTime="2026-07-07 15:07:49.857570159 +0000 UTC m=+437.654106886"
Jul 07 15:07:53.295033 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:53.294992    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fjg5j"]
Jul 07 15:07:53.299019 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:53.299002    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fjg5j"
Jul 07 15:07:53.304272 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:53.304252    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-c48bh\""
Jul 07 15:07:53.311781 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:53.311755    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fjg5j"]
Jul 07 15:07:53.438913 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:53.438877    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j68xn\" (UniqueName: \"kubernetes.io/projected/d811e5a0-0ee7-4c2e-98c6-837d1349ff0c-kube-api-access-j68xn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fjg5j\" (UID: \"d811e5a0-0ee7-4c2e-98c6-837d1349ff0c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fjg5j"
Jul 07 15:07:53.439081 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:53.438930    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d811e5a0-0ee7-4c2e-98c6-837d1349ff0c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fjg5j\" (UID: \"d811e5a0-0ee7-4c2e-98c6-837d1349ff0c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fjg5j"
Jul 07 15:07:53.540294 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:53.540247    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j68xn\" (UniqueName: \"kubernetes.io/projected/d811e5a0-0ee7-4c2e-98c6-837d1349ff0c-kube-api-access-j68xn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fjg5j\" (UID: \"d811e5a0-0ee7-4c2e-98c6-837d1349ff0c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fjg5j"
Jul 07 15:07:53.540471 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:53.540331    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d811e5a0-0ee7-4c2e-98c6-837d1349ff0c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fjg5j\" (UID: \"d811e5a0-0ee7-4c2e-98c6-837d1349ff0c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fjg5j"
Jul 07 15:07:53.540724 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:53.540702    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d811e5a0-0ee7-4c2e-98c6-837d1349ff0c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fjg5j\" (UID: \"d811e5a0-0ee7-4c2e-98c6-837d1349ff0c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fjg5j"
Jul 07 15:07:53.551691 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:53.551666    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j68xn\" (UniqueName: \"kubernetes.io/projected/d811e5a0-0ee7-4c2e-98c6-837d1349ff0c-kube-api-access-j68xn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fjg5j\" (UID: \"d811e5a0-0ee7-4c2e-98c6-837d1349ff0c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fjg5j"
Jul 07 15:07:53.611576 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:53.611543    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fjg5j"
Jul 07 15:07:53.760389 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:53.760361    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fjg5j"]
Jul 07 15:07:53.762132 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:07:53.762103    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd811e5a0_0ee7_4c2e_98c6_837d1349ff0c.slice/crio-f51c6caa86a11502cd1f4b0b3be19f8ab627cf546e9d9808afbc17a7aef773c5 WatchSource:0}: Error finding container f51c6caa86a11502cd1f4b0b3be19f8ab627cf546e9d9808afbc17a7aef773c5: Status 404 returned error can't find the container with id f51c6caa86a11502cd1f4b0b3be19f8ab627cf546e9d9808afbc17a7aef773c5
Jul 07 15:07:53.853444 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:53.853346    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fjg5j" event={"ID":"d811e5a0-0ee7-4c2e-98c6-837d1349ff0c","Type":"ContainerStarted","Data":"f51c6caa86a11502cd1f4b0b3be19f8ab627cf546e9d9808afbc17a7aef773c5"}
Jul 07 15:07:53.854929 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:53.854905    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t6tmh" event={"ID":"539a21f4-9e00-4fa5-bff8-2f5388906b1b","Type":"ContainerStarted","Data":"ccd6e7fc124f44662ffbc31abd65c3a7192d0df8485374a91b563cd53d2d6034"}
Jul 07 15:07:53.855047 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:53.855029    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t6tmh"
Jul 07 15:07:53.879013 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:53.878960    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t6tmh" podStartSLOduration=2.027671422 podStartE2EDuration="5.878944314s" totalImagesPullingTime="3.851272892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:07:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:07:53.878320984 +0000 UTC m=+441.674857712" watchObservedRunningTime="2026-07-07 15:07:53.878944314 +0000 UTC m=+441.675481042"
Jul 07 15:07:57.820990 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:07:57.820960    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hj44h"
Jul 07 15:08:00.838629 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:00.838583    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k4dqc"
Jul 07 15:08:01.899714 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:01.899619    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fjg5j" event={"ID":"d811e5a0-0ee7-4c2e-98c6-837d1349ff0c","Type":"ContainerStarted","Data":"7dad59008f7141313a0946a3d2f236c2d9604012ed66b370d0a92a94d20ac077"}
Jul 07 15:08:01.900117 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:01.899788    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fjg5j"
Jul 07 15:08:01.946486 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:01.946433    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fjg5j" podStartSLOduration=1.107449708 podStartE2EDuration="8.946416719s" totalImagesPullingTime="7.838967011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:07:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:08:01.946273827 +0000 UTC m=+449.742810569" watchObservedRunningTime="2026-07-07 15:08:01.946416719 +0000 UTC m=+449.742953492"
Jul 07 15:08:04.862131 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:04.862095    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-t6tmh"
Jul 07 15:08:12.907252 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:12.907219    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fjg5j"
Jul 07 15:08:13.937696 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:13.937661    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cw8p2"]
Jul 07 15:08:13.941269 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:13.941249    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cw8p2"
Jul 07 15:08:13.956331 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:13.956309    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cw8p2"]
Jul 07 15:08:14.000225 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.000184    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1e9bba4f-21f9-47e9-92c3-6794fe615634-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-cw8p2\" (UID: \"1e9bba4f-21f9-47e9-92c3-6794fe615634\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cw8p2"
Jul 07 15:08:14.000413 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.000238    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9jff\" (UniqueName: \"kubernetes.io/projected/1e9bba4f-21f9-47e9-92c3-6794fe615634-kube-api-access-f9jff\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-cw8p2\" (UID: \"1e9bba4f-21f9-47e9-92c3-6794fe615634\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cw8p2"
Jul 07 15:08:14.100699 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.100653    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1e9bba4f-21f9-47e9-92c3-6794fe615634-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-cw8p2\" (UID: \"1e9bba4f-21f9-47e9-92c3-6794fe615634\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cw8p2"
Jul 07 15:08:14.100699 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.100704    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9jff\" (UniqueName: \"kubernetes.io/projected/1e9bba4f-21f9-47e9-92c3-6794fe615634-kube-api-access-f9jff\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-cw8p2\" (UID: \"1e9bba4f-21f9-47e9-92c3-6794fe615634\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cw8p2"
Jul 07 15:08:14.101076 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.101053    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1e9bba4f-21f9-47e9-92c3-6794fe615634-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-cw8p2\" (UID: \"1e9bba4f-21f9-47e9-92c3-6794fe615634\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cw8p2"
Jul 07 15:08:14.112349 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.112323    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9jff\" (UniqueName: \"kubernetes.io/projected/1e9bba4f-21f9-47e9-92c3-6794fe615634-kube-api-access-f9jff\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-cw8p2\" (UID: \"1e9bba4f-21f9-47e9-92c3-6794fe615634\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cw8p2"
Jul 07 15:08:14.251536 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.251433    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cw8p2"
Jul 07 15:08:14.435132 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.435107    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cw8p2"]
Jul 07 15:08:14.437221 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:08:14.437192    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e9bba4f_21f9_47e9_92c3_6794fe615634.slice/crio-4fc8163c6ac5a800d8d712f69c66952e195da17550c12c0654e0d4cb294f6359 WatchSource:0}: Error finding container 4fc8163c6ac5a800d8d712f69c66952e195da17550c12c0654e0d4cb294f6359: Status 404 returned error can't find the container with id 4fc8163c6ac5a800d8d712f69c66952e195da17550c12c0654e0d4cb294f6359
Jul 07 15:08:14.730645 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.730610    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cw8p2"]
Jul 07 15:08:14.736134 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.736106    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cw8p2"]
Jul 07 15:08:14.746972 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.746944    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fjg5j"]
Jul 07 15:08:14.747164 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.747128    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fjg5j" podUID="d811e5a0-0ee7-4c2e-98c6-837d1349ff0c" containerName="manager" containerID="cri-o://7dad59008f7141313a0946a3d2f236c2d9604012ed66b370d0a92a94d20ac077" gracePeriod=2
Jul 07 15:08:14.763706 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.763676    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fjg5j"]
Jul 07 15:08:14.766310 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.766282    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-d28bs"]
Jul 07 15:08:14.766886 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.766863    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d811e5a0-0ee7-4c2e-98c6-837d1349ff0c" containerName="manager"
Jul 07 15:08:14.766886 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.766889    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d811e5a0-0ee7-4c2e-98c6-837d1349ff0c" containerName="manager"
Jul 07 15:08:14.767047 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.766931    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e9bba4f-21f9-47e9-92c3-6794fe615634" containerName="manager"
Jul 07 15:08:14.767047 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.766942    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e9bba4f-21f9-47e9-92c3-6794fe615634" containerName="manager"
Jul 07 15:08:14.767232 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.767212    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d811e5a0-0ee7-4c2e-98c6-837d1349ff0c" containerName="manager"
Jul 07 15:08:14.767280 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.767241    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e9bba4f-21f9-47e9-92c3-6794fe615634" containerName="manager"
Jul 07 15:08:14.770062 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:08:14.770021    2567 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"d811e5a0-0ee7-4c2e-98c6-837d1349ff0c\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-fjg5j\": pods \"kuadrant-operator-controller-manager-8684c9ff49-fjg5j\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fjg5j"
Jul 07 15:08:14.770257 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.770239    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-d28bs"
Jul 07 15:08:14.773033 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:08:14.773003    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fjg5j\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="d811e5a0-0ee7-4c2e-98c6-837d1349ff0c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fjg5j"
Jul 07 15:08:14.781139 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.781112    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjkbq"]
Jul 07 15:08:14.784721 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.784693    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k4dqc"]
Jul 07 15:08:14.785392 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.784890    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjkbq"
Jul 07 15:08:14.785392 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.784913    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k4dqc" podUID="f36e272b-dd1e-4d7f-b50b-a0f89bd69412" containerName="manager" containerID="cri-o://accb8bc8a5d9baeecb9fc9d56bfae5222c0b4f0738158fd43f32dc3f00bb70b9" gracePeriod=2
Jul 07 15:08:14.785650 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.785627    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k4dqc"]
Jul 07 15:08:14.797128 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.797084    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-d28bs"]
Jul 07 15:08:14.798981 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.798944    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjkbq"]
Jul 07 15:08:14.806154 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.806124    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/198bab9f-1bd4-4178-a321-9856c02dfcf4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kjkbq\" (UID: \"198bab9f-1bd4-4178-a321-9856c02dfcf4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjkbq"
Jul 07 15:08:14.806298 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.806179    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnr2l\" (UniqueName: \"kubernetes.io/projected/198bab9f-1bd4-4178-a321-9856c02dfcf4-kube-api-access-nnr2l\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kjkbq\" (UID: \"198bab9f-1bd4-4178-a321-9856c02dfcf4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjkbq"
Jul 07 15:08:14.806367 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.806336    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9p7fc\" (UniqueName: \"kubernetes.io/projected/ef55cae6-f66d-4a79-9cfb-2d3764639f28-kube-api-access-9p7fc\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-d28bs\" (UID: \"ef55cae6-f66d-4a79-9cfb-2d3764639f28\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-d28bs"
Jul 07 15:08:14.806420 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.806382    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef55cae6-f66d-4a79-9cfb-2d3764639f28-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-d28bs\" (UID: \"ef55cae6-f66d-4a79-9cfb-2d3764639f28\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-d28bs"
Jul 07 15:08:14.845907 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:08:14.845870    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-k4dqc\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="f36e272b-dd1e-4d7f-b50b-a0f89bd69412" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k4dqc"
Jul 07 15:08:14.907192 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.907159    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/198bab9f-1bd4-4178-a321-9856c02dfcf4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kjkbq\" (UID: \"198bab9f-1bd4-4178-a321-9856c02dfcf4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjkbq"
Jul 07 15:08:14.907375 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.907207    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnr2l\" (UniqueName: \"kubernetes.io/projected/198bab9f-1bd4-4178-a321-9856c02dfcf4-kube-api-access-nnr2l\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kjkbq\" (UID: \"198bab9f-1bd4-4178-a321-9856c02dfcf4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjkbq"
Jul 07 15:08:14.907375 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.907275    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9p7fc\" (UniqueName: \"kubernetes.io/projected/ef55cae6-f66d-4a79-9cfb-2d3764639f28-kube-api-access-9p7fc\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-d28bs\" (UID: \"ef55cae6-f66d-4a79-9cfb-2d3764639f28\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-d28bs"
Jul 07 15:08:14.907375 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.907306    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef55cae6-f66d-4a79-9cfb-2d3764639f28-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-d28bs\" (UID: \"ef55cae6-f66d-4a79-9cfb-2d3764639f28\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-d28bs"
Jul 07 15:08:14.907634 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.907609    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/198bab9f-1bd4-4178-a321-9856c02dfcf4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kjkbq\" (UID: \"198bab9f-1bd4-4178-a321-9856c02dfcf4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjkbq"
Jul 07 15:08:14.907634 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.907632    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef55cae6-f66d-4a79-9cfb-2d3764639f28-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-d28bs\" (UID: \"ef55cae6-f66d-4a79-9cfb-2d3764639f28\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-d28bs"
Jul 07 15:08:14.922299 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.922269    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnr2l\" (UniqueName: \"kubernetes.io/projected/198bab9f-1bd4-4178-a321-9856c02dfcf4-kube-api-access-nnr2l\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kjkbq\" (UID: \"198bab9f-1bd4-4178-a321-9856c02dfcf4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjkbq"
Jul 07 15:08:14.922488 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.922376    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9p7fc\" (UniqueName: \"kubernetes.io/projected/ef55cae6-f66d-4a79-9cfb-2d3764639f28-kube-api-access-9p7fc\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-d28bs\" (UID: \"ef55cae6-f66d-4a79-9cfb-2d3764639f28\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-d28bs"
Jul 07 15:08:14.967271 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.967243    2567 generic.go:356] "Generic (PLEG): container finished" podID="f36e272b-dd1e-4d7f-b50b-a0f89bd69412" containerID="accb8bc8a5d9baeecb9fc9d56bfae5222c0b4f0738158fd43f32dc3f00bb70b9" exitCode=0
Jul 07 15:08:14.969339 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:14.969316    2567 generic.go:356] "Generic (PLEG): container finished" podID="d811e5a0-0ee7-4c2e-98c6-837d1349ff0c" containerID="7dad59008f7141313a0946a3d2f236c2d9604012ed66b370d0a92a94d20ac077" exitCode=0
Jul 07 15:08:14.971568 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:08:14.971540    2567 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cw8p2" container="manager"
Jul 07 15:08:14.975014 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:08:14.974984    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-k4dqc\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="f36e272b-dd1e-4d7f-b50b-a0f89bd69412" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k4dqc"
Jul 07 15:08:15.023128 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:15.023099    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fjg5j"
Jul 07 15:08:15.025862 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:08:15.025795    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-k4dqc\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="f36e272b-dd1e-4d7f-b50b-a0f89bd69412" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k4dqc"
Jul 07 15:08:15.033537 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:15.033511    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k4dqc"
Jul 07 15:08:15.036268 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:08:15.036241    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-k4dqc\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="f36e272b-dd1e-4d7f-b50b-a0f89bd69412" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k4dqc"
Jul 07 15:08:15.109272 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:15.109224    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwspp\" (UniqueName: \"kubernetes.io/projected/f36e272b-dd1e-4d7f-b50b-a0f89bd69412-kube-api-access-lwspp\") pod \"f36e272b-dd1e-4d7f-b50b-a0f89bd69412\" (UID: \"f36e272b-dd1e-4d7f-b50b-a0f89bd69412\") "
Jul 07 15:08:15.109272 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:15.109277    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j68xn\" (UniqueName: \"kubernetes.io/projected/d811e5a0-0ee7-4c2e-98c6-837d1349ff0c-kube-api-access-j68xn\") pod \"d811e5a0-0ee7-4c2e-98c6-837d1349ff0c\" (UID: \"d811e5a0-0ee7-4c2e-98c6-837d1349ff0c\") "
Jul 07 15:08:15.109528 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:15.109297    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d811e5a0-0ee7-4c2e-98c6-837d1349ff0c-extensions-socket-volume\") pod \"d811e5a0-0ee7-4c2e-98c6-837d1349ff0c\" (UID: \"d811e5a0-0ee7-4c2e-98c6-837d1349ff0c\") "
Jul 07 15:08:15.109765 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:15.109729    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d811e5a0-0ee7-4c2e-98c6-837d1349ff0c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d811e5a0-0ee7-4c2e-98c6-837d1349ff0c" (UID: "d811e5a0-0ee7-4c2e-98c6-837d1349ff0c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:08:15.111543 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:15.111518    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f36e272b-dd1e-4d7f-b50b-a0f89bd69412-kube-api-access-lwspp" (OuterVolumeSpecName: "kube-api-access-lwspp") pod "f36e272b-dd1e-4d7f-b50b-a0f89bd69412" (UID: "f36e272b-dd1e-4d7f-b50b-a0f89bd69412"). InnerVolumeSpecName "kube-api-access-lwspp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:08:15.111610 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:15.111524    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d811e5a0-0ee7-4c2e-98c6-837d1349ff0c-kube-api-access-j68xn" (OuterVolumeSpecName: "kube-api-access-j68xn") pod "d811e5a0-0ee7-4c2e-98c6-837d1349ff0c" (UID: "d811e5a0-0ee7-4c2e-98c6-837d1349ff0c"). InnerVolumeSpecName "kube-api-access-j68xn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:08:15.176041 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:15.176006    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-d28bs"
Jul 07 15:08:15.182198 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:15.182169    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjkbq"
Jul 07 15:08:15.211521 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:15.211316    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwspp\" (UniqueName: \"kubernetes.io/projected/f36e272b-dd1e-4d7f-b50b-a0f89bd69412-kube-api-access-lwspp\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:08:15.211521 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:15.211355    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j68xn\" (UniqueName: \"kubernetes.io/projected/d811e5a0-0ee7-4c2e-98c6-837d1349ff0c-kube-api-access-j68xn\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:08:15.211521 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:15.211372    2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d811e5a0-0ee7-4c2e-98c6-837d1349ff0c-extensions-socket-volume\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:08:15.358935 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:15.358904    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-d28bs"]
Jul 07 15:08:15.361472 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:08:15.361444    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef55cae6_f66d_4a79_9cfb_2d3764639f28.slice/crio-c1e31ce8e7fc0a54768b286c1328938370a6f0c11a245bb6e79eddac7855aba9 WatchSource:0}: Error finding container c1e31ce8e7fc0a54768b286c1328938370a6f0c11a245bb6e79eddac7855aba9: Status 404 returned error can't find the container with id c1e31ce8e7fc0a54768b286c1328938370a6f0c11a245bb6e79eddac7855aba9
Jul 07 15:08:15.396154 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:15.396121    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjkbq"]
Jul 07 15:08:15.400085 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:08:15.400051    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod198bab9f_1bd4_4178_a321_9856c02dfcf4.slice/crio-adb7ce784cba6d82a21761d6f2a5e9966dd5263bc183717c0f0656697934c029 WatchSource:0}: Error finding container adb7ce784cba6d82a21761d6f2a5e9966dd5263bc183717c0f0656697934c029: Status 404 returned error can't find the container with id adb7ce784cba6d82a21761d6f2a5e9966dd5263bc183717c0f0656697934c029
Jul 07 15:08:15.978131 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:15.978091    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cw8p2" podUID="1e9bba4f-21f9-47e9-92c3-6794fe615634" containerName="manager" containerID="cri-o://9b56757621a1b26caa7ddc102f4578d906c295dbb1ab36bbc43199608817c7fb" gracePeriod=2
Jul 07 15:08:15.979875 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:15.979825    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-d28bs" event={"ID":"ef55cae6-f66d-4a79-9cfb-2d3764639f28","Type":"ContainerStarted","Data":"9afbf4b49f4c4425205483986a2b91f30400dd4c4e4bb7100ebe4c354ba4e7fd"}
Jul 07 15:08:15.980019 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:15.979890    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-d28bs" event={"ID":"ef55cae6-f66d-4a79-9cfb-2d3764639f28","Type":"ContainerStarted","Data":"c1e31ce8e7fc0a54768b286c1328938370a6f0c11a245bb6e79eddac7855aba9"}
Jul 07 15:08:15.980019 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:15.979921    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-d28bs"
Jul 07 15:08:15.981897 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:15.981869    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjkbq" event={"ID":"198bab9f-1bd4-4178-a321-9856c02dfcf4","Type":"ContainerStarted","Data":"b37bfc96969ff292d02f89c6f9dbc309478426937a85b013c1c32140ed7ef904"}
Jul 07 15:08:15.982029 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:15.981924    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjkbq" event={"ID":"198bab9f-1bd4-4178-a321-9856c02dfcf4","Type":"ContainerStarted","Data":"adb7ce784cba6d82a21761d6f2a5e9966dd5263bc183717c0f0656697934c029"}
Jul 07 15:08:15.982029 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:15.981976    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjkbq"
Jul 07 15:08:15.982029 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:08:15.981986    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-k4dqc\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="f36e272b-dd1e-4d7f-b50b-a0f89bd69412" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k4dqc"
Jul 07 15:08:15.983177 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:15.983160    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k4dqc"
Jul 07 15:08:15.983284 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:15.983163    2567 scope.go:117] "RemoveContainer" containerID="accb8bc8a5d9baeecb9fc9d56bfae5222c0b4f0738158fd43f32dc3f00bb70b9"
Jul 07 15:08:15.984780 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:15.984762    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fjg5j"
Jul 07 15:08:15.985591 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:08:15.985555    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-cw8p2\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="1e9bba4f-21f9-47e9-92c3-6794fe615634" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cw8p2"
Jul 07 15:08:16.022158 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:16.022108    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjkbq" podStartSLOduration=2.022092037 podStartE2EDuration="2.022092037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:08:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:08:16.020746487 +0000 UTC m=+463.817283216" watchObservedRunningTime="2026-07-07 15:08:16.022092037 +0000 UTC m=+463.818628765"
Jul 07 15:08:16.023610 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:08:16.023584    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-k4dqc\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="f36e272b-dd1e-4d7f-b50b-a0f89bd69412" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k4dqc"
Jul 07 15:08:16.026547 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:08:16.026505    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-cw8p2\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="1e9bba4f-21f9-47e9-92c3-6794fe615634" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cw8p2"
Jul 07 15:08:16.029173 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:16.029158    2567 scope.go:117] "RemoveContainer" containerID="7dad59008f7141313a0946a3d2f236c2d9604012ed66b370d0a92a94d20ac077"
Jul 07 15:08:16.083552 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:08:16.083509    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-k4dqc\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="f36e272b-dd1e-4d7f-b50b-a0f89bd69412" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k4dqc"
Jul 07 15:08:16.087001 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:08:16.086968    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-cw8p2\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="1e9bba4f-21f9-47e9-92c3-6794fe615634" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cw8p2"
Jul 07 15:08:16.089739 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:16.089698    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-d28bs" podStartSLOduration=2.089684101 podStartE2EDuration="2.089684101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:08:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:08:16.080393674 +0000 UTC m=+463.876930403" watchObservedRunningTime="2026-07-07 15:08:16.089684101 +0000 UTC m=+463.886220828"
Jul 07 15:08:16.245493 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:16.245430    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cw8p2"
Jul 07 15:08:16.250232 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:08:16.250201    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-k4dqc\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="f36e272b-dd1e-4d7f-b50b-a0f89bd69412" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k4dqc"
Jul 07 15:08:16.253769 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:08:16.253735    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-cw8p2\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="1e9bba4f-21f9-47e9-92c3-6794fe615634" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cw8p2"
Jul 07 15:08:16.321149 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:16.321113    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1e9bba4f-21f9-47e9-92c3-6794fe615634-extensions-socket-volume\") pod \"1e9bba4f-21f9-47e9-92c3-6794fe615634\" (UID: \"1e9bba4f-21f9-47e9-92c3-6794fe615634\") "
Jul 07 15:08:16.321342 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:16.321168    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9jff\" (UniqueName: \"kubernetes.io/projected/1e9bba4f-21f9-47e9-92c3-6794fe615634-kube-api-access-f9jff\") pod \"1e9bba4f-21f9-47e9-92c3-6794fe615634\" (UID: \"1e9bba4f-21f9-47e9-92c3-6794fe615634\") "
Jul 07 15:08:16.321591 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:16.321562    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e9bba4f-21f9-47e9-92c3-6794fe615634-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1e9bba4f-21f9-47e9-92c3-6794fe615634" (UID: "1e9bba4f-21f9-47e9-92c3-6794fe615634"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:08:16.323598 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:16.323571    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e9bba4f-21f9-47e9-92c3-6794fe615634-kube-api-access-f9jff" (OuterVolumeSpecName: "kube-api-access-f9jff") pod "1e9bba4f-21f9-47e9-92c3-6794fe615634" (UID: "1e9bba4f-21f9-47e9-92c3-6794fe615634"). InnerVolumeSpecName "kube-api-access-f9jff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:08:16.422160 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:16.422121    2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1e9bba4f-21f9-47e9-92c3-6794fe615634-extensions-socket-volume\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:08:16.422160 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:16.422163    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9jff\" (UniqueName: \"kubernetes.io/projected/1e9bba4f-21f9-47e9-92c3-6794fe615634-kube-api-access-f9jff\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:08:16.778223 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:16.778189    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e9bba4f-21f9-47e9-92c3-6794fe615634" path="/var/lib/kubelet/pods/1e9bba4f-21f9-47e9-92c3-6794fe615634/volumes"
Jul 07 15:08:16.778708 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:16.778688    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d811e5a0-0ee7-4c2e-98c6-837d1349ff0c" path="/var/lib/kubelet/pods/d811e5a0-0ee7-4c2e-98c6-837d1349ff0c/volumes"
Jul 07 15:08:16.779126 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:16.779099    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f36e272b-dd1e-4d7f-b50b-a0f89bd69412" path="/var/lib/kubelet/pods/f36e272b-dd1e-4d7f-b50b-a0f89bd69412/volumes"
Jul 07 15:08:16.992015 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:16.991982    2567 generic.go:356] "Generic (PLEG): container finished" podID="1e9bba4f-21f9-47e9-92c3-6794fe615634" containerID="9b56757621a1b26caa7ddc102f4578d906c295dbb1ab36bbc43199608817c7fb" exitCode=0
Jul 07 15:08:16.992483 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:16.992028    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cw8p2"
Jul 07 15:08:16.992483 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:16.992088    2567 scope.go:117] "RemoveContainer" containerID="9b56757621a1b26caa7ddc102f4578d906c295dbb1ab36bbc43199608817c7fb"
Jul 07 15:08:16.998208 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:08:16.998167    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-cw8p2\" is forbidden: User \"system:node:ip-10-0-138-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-233.ec2.internal' and this object" podUID="1e9bba4f-21f9-47e9-92c3-6794fe615634" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cw8p2"
Jul 07 15:08:17.001689 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:17.001666    2567 scope.go:117] "RemoveContainer" containerID="9b56757621a1b26caa7ddc102f4578d906c295dbb1ab36bbc43199608817c7fb"
Jul 07 15:08:17.002006 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:08:17.001985    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b56757621a1b26caa7ddc102f4578d906c295dbb1ab36bbc43199608817c7fb\": container with ID starting with 9b56757621a1b26caa7ddc102f4578d906c295dbb1ab36bbc43199608817c7fb not found: ID does not exist" containerID="9b56757621a1b26caa7ddc102f4578d906c295dbb1ab36bbc43199608817c7fb"
Jul 07 15:08:17.002075 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:17.002017    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b56757621a1b26caa7ddc102f4578d906c295dbb1ab36bbc43199608817c7fb"} err="failed to get container status \"9b56757621a1b26caa7ddc102f4578d906c295dbb1ab36bbc43199608817c7fb\": rpc error: code = NotFound desc = could not find container \"9b56757621a1b26caa7ddc102f4578d906c295dbb1ab36bbc43199608817c7fb\": container with ID starting with 9b56757621a1b26caa7ddc102f4578d906c295dbb1ab36bbc43199608817c7fb not found: ID does not exist"
Jul 07 15:08:26.997415 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:26.997387    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-d28bs"
Jul 07 15:08:26.997926 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:26.997464    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjkbq"
Jul 07 15:08:27.093366 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:27.093335    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjkbq"]
Jul 07 15:08:27.093551 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:27.093525    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjkbq" podUID="198bab9f-1bd4-4178-a321-9856c02dfcf4" containerName="manager" containerID="cri-o://b37bfc96969ff292d02f89c6f9dbc309478426937a85b013c1c32140ed7ef904" gracePeriod=10
Jul 07 15:08:27.349193 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:27.349162    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjkbq"
Jul 07 15:08:27.410408 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:27.410375    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/198bab9f-1bd4-4178-a321-9856c02dfcf4-extensions-socket-volume\") pod \"198bab9f-1bd4-4178-a321-9856c02dfcf4\" (UID: \"198bab9f-1bd4-4178-a321-9856c02dfcf4\") "
Jul 07 15:08:27.410561 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:27.410424    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nnr2l\" (UniqueName: \"kubernetes.io/projected/198bab9f-1bd4-4178-a321-9856c02dfcf4-kube-api-access-nnr2l\") pod \"198bab9f-1bd4-4178-a321-9856c02dfcf4\" (UID: \"198bab9f-1bd4-4178-a321-9856c02dfcf4\") "
Jul 07 15:08:27.410756 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:27.410733    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/198bab9f-1bd4-4178-a321-9856c02dfcf4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "198bab9f-1bd4-4178-a321-9856c02dfcf4" (UID: "198bab9f-1bd4-4178-a321-9856c02dfcf4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:08:27.412645 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:27.412619    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/198bab9f-1bd4-4178-a321-9856c02dfcf4-kube-api-access-nnr2l" (OuterVolumeSpecName: "kube-api-access-nnr2l") pod "198bab9f-1bd4-4178-a321-9856c02dfcf4" (UID: "198bab9f-1bd4-4178-a321-9856c02dfcf4"). InnerVolumeSpecName "kube-api-access-nnr2l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:08:27.473311 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:27.473282    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hp2v"]
Jul 07 15:08:27.473651 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:27.473632    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f36e272b-dd1e-4d7f-b50b-a0f89bd69412" containerName="manager"
Jul 07 15:08:27.473729 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:27.473653    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f36e272b-dd1e-4d7f-b50b-a0f89bd69412" containerName="manager"
Jul 07 15:08:27.473729 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:27.473704    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="198bab9f-1bd4-4178-a321-9856c02dfcf4" containerName="manager"
Jul 07 15:08:27.473729 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:27.473714    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="198bab9f-1bd4-4178-a321-9856c02dfcf4" containerName="manager"
Jul 07 15:08:27.473948 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:27.473861    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="198bab9f-1bd4-4178-a321-9856c02dfcf4" containerName="manager"
Jul 07 15:08:27.473948 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:27.473882    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f36e272b-dd1e-4d7f-b50b-a0f89bd69412" containerName="manager"
Jul 07 15:08:27.476679 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:27.476659    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hp2v"
Jul 07 15:08:27.507765 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:27.507728    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hp2v"]
Jul 07 15:08:27.511028 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:27.511002    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z76fb\" (UniqueName: \"kubernetes.io/projected/8929b62d-4b20-4bd3-bfa0-5ed756f58425-kube-api-access-z76fb\") pod \"kuadrant-operator-controller-manager-86878c769f-9hp2v\" (UID: \"8929b62d-4b20-4bd3-bfa0-5ed756f58425\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hp2v"
Jul 07 15:08:27.511151 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:27.511035    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8929b62d-4b20-4bd3-bfa0-5ed756f58425-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9hp2v\" (UID: \"8929b62d-4b20-4bd3-bfa0-5ed756f58425\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hp2v"
Jul 07 15:08:27.511191 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:27.511168    2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/198bab9f-1bd4-4178-a321-9856c02dfcf4-extensions-socket-volume\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:08:27.511222 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:27.511191    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnr2l\" (UniqueName: \"kubernetes.io/projected/198bab9f-1bd4-4178-a321-9856c02dfcf4-kube-api-access-nnr2l\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:08:27.612494 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:27.612459    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z76fb\" (UniqueName: \"kubernetes.io/projected/8929b62d-4b20-4bd3-bfa0-5ed756f58425-kube-api-access-z76fb\") pod \"kuadrant-operator-controller-manager-86878c769f-9hp2v\" (UID: \"8929b62d-4b20-4bd3-bfa0-5ed756f58425\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hp2v"
Jul 07 15:08:27.612494 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:27.612497    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8929b62d-4b20-4bd3-bfa0-5ed756f58425-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9hp2v\" (UID: \"8929b62d-4b20-4bd3-bfa0-5ed756f58425\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hp2v"
Jul 07 15:08:27.612817 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:27.612800    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8929b62d-4b20-4bd3-bfa0-5ed756f58425-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9hp2v\" (UID: \"8929b62d-4b20-4bd3-bfa0-5ed756f58425\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hp2v"
Jul 07 15:08:27.624359 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:27.624335    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z76fb\" (UniqueName: \"kubernetes.io/projected/8929b62d-4b20-4bd3-bfa0-5ed756f58425-kube-api-access-z76fb\") pod \"kuadrant-operator-controller-manager-86878c769f-9hp2v\" (UID: \"8929b62d-4b20-4bd3-bfa0-5ed756f58425\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hp2v"
Jul 07 15:08:27.786355 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:27.786320    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hp2v"
Jul 07 15:08:27.930024 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:27.929998    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hp2v"]
Jul 07 15:08:27.932044 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:08:27.932015    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8929b62d_4b20_4bd3_bfa0_5ed756f58425.slice/crio-7163ab1c5d2ba08987c18a5712866028b3d32f8e11caaa9c414e53233e29327f WatchSource:0}: Error finding container 7163ab1c5d2ba08987c18a5712866028b3d32f8e11caaa9c414e53233e29327f: Status 404 returned error can't find the container with id 7163ab1c5d2ba08987c18a5712866028b3d32f8e11caaa9c414e53233e29327f
Jul 07 15:08:28.051390 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:28.051354    2567 generic.go:356] "Generic (PLEG): container finished" podID="198bab9f-1bd4-4178-a321-9856c02dfcf4" containerID="b37bfc96969ff292d02f89c6f9dbc309478426937a85b013c1c32140ed7ef904" exitCode=0
Jul 07 15:08:28.051869 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:28.051424    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjkbq"
Jul 07 15:08:28.051869 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:28.051434    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjkbq" event={"ID":"198bab9f-1bd4-4178-a321-9856c02dfcf4","Type":"ContainerDied","Data":"b37bfc96969ff292d02f89c6f9dbc309478426937a85b013c1c32140ed7ef904"}
Jul 07 15:08:28.051869 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:28.051500    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjkbq" event={"ID":"198bab9f-1bd4-4178-a321-9856c02dfcf4","Type":"ContainerDied","Data":"adb7ce784cba6d82a21761d6f2a5e9966dd5263bc183717c0f0656697934c029"}
Jul 07 15:08:28.051869 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:28.051535    2567 scope.go:117] "RemoveContainer" containerID="b37bfc96969ff292d02f89c6f9dbc309478426937a85b013c1c32140ed7ef904"
Jul 07 15:08:28.053361 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:28.053335    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hp2v" event={"ID":"8929b62d-4b20-4bd3-bfa0-5ed756f58425","Type":"ContainerStarted","Data":"00f776c488a87e94cdef526defbff66055a952fcfca8b87df2e4951e30916a0c"}
Jul 07 15:08:28.053483 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:28.053390    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hp2v" event={"ID":"8929b62d-4b20-4bd3-bfa0-5ed756f58425","Type":"ContainerStarted","Data":"7163ab1c5d2ba08987c18a5712866028b3d32f8e11caaa9c414e53233e29327f"}
Jul 07 15:08:28.053483 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:28.053443    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hp2v"
Jul 07 15:08:28.060679 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:28.060663    2567 scope.go:117] "RemoveContainer" containerID="b37bfc96969ff292d02f89c6f9dbc309478426937a85b013c1c32140ed7ef904"
Jul 07 15:08:28.060980 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:08:28.060959    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b37bfc96969ff292d02f89c6f9dbc309478426937a85b013c1c32140ed7ef904\": container with ID starting with b37bfc96969ff292d02f89c6f9dbc309478426937a85b013c1c32140ed7ef904 not found: ID does not exist" containerID="b37bfc96969ff292d02f89c6f9dbc309478426937a85b013c1c32140ed7ef904"
Jul 07 15:08:28.061041 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:28.060989    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b37bfc96969ff292d02f89c6f9dbc309478426937a85b013c1c32140ed7ef904"} err="failed to get container status \"b37bfc96969ff292d02f89c6f9dbc309478426937a85b013c1c32140ed7ef904\": rpc error: code = NotFound desc = could not find container \"b37bfc96969ff292d02f89c6f9dbc309478426937a85b013c1c32140ed7ef904\": container with ID starting with b37bfc96969ff292d02f89c6f9dbc309478426937a85b013c1c32140ed7ef904 not found: ID does not exist"
Jul 07 15:08:28.076939 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:28.076907    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjkbq"]
Jul 07 15:08:28.079845 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:28.079799    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjkbq"]
Jul 07 15:08:28.101593 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:28.101523    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hp2v" podStartSLOduration=1.10150483 podStartE2EDuration="1.10150483s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:08:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:08:28.09998046 +0000 UTC m=+475.896517188" watchObservedRunningTime="2026-07-07 15:08:28.10150483 +0000 UTC m=+475.898041595"
Jul 07 15:08:28.777895 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:28.777860    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="198bab9f-1bd4-4178-a321-9856c02dfcf4" path="/var/lib/kubelet/pods/198bab9f-1bd4-4178-a321-9856c02dfcf4/volumes"
Jul 07 15:08:39.061642 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:39.061608    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hp2v"
Jul 07 15:08:39.136012 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:39.135969    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-d28bs"]
Jul 07 15:08:39.136219 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:39.136174    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-d28bs" podUID="ef55cae6-f66d-4a79-9cfb-2d3764639f28" containerName="manager" containerID="cri-o://9afbf4b49f4c4425205483986a2b91f30400dd4c4e4bb7100ebe4c354ba4e7fd" gracePeriod=10
Jul 07 15:08:39.399659 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:39.399628    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-d28bs"
Jul 07 15:08:39.519153 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:39.519118    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9p7fc\" (UniqueName: \"kubernetes.io/projected/ef55cae6-f66d-4a79-9cfb-2d3764639f28-kube-api-access-9p7fc\") pod \"ef55cae6-f66d-4a79-9cfb-2d3764639f28\" (UID: \"ef55cae6-f66d-4a79-9cfb-2d3764639f28\") "
Jul 07 15:08:39.519358 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:39.519180    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef55cae6-f66d-4a79-9cfb-2d3764639f28-extensions-socket-volume\") pod \"ef55cae6-f66d-4a79-9cfb-2d3764639f28\" (UID: \"ef55cae6-f66d-4a79-9cfb-2d3764639f28\") "
Jul 07 15:08:39.519624 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:39.519588    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef55cae6-f66d-4a79-9cfb-2d3764639f28-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ef55cae6-f66d-4a79-9cfb-2d3764639f28" (UID: "ef55cae6-f66d-4a79-9cfb-2d3764639f28"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:08:39.521502 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:39.521478    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef55cae6-f66d-4a79-9cfb-2d3764639f28-kube-api-access-9p7fc" (OuterVolumeSpecName: "kube-api-access-9p7fc") pod "ef55cae6-f66d-4a79-9cfb-2d3764639f28" (UID: "ef55cae6-f66d-4a79-9cfb-2d3764639f28"). InnerVolumeSpecName "kube-api-access-9p7fc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:08:39.620392 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:39.620352    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9p7fc\" (UniqueName: \"kubernetes.io/projected/ef55cae6-f66d-4a79-9cfb-2d3764639f28-kube-api-access-9p7fc\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:08:39.620392 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:39.620386    2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef55cae6-f66d-4a79-9cfb-2d3764639f28-extensions-socket-volume\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:08:40.112019 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:40.111985    2567 generic.go:356] "Generic (PLEG): container finished" podID="ef55cae6-f66d-4a79-9cfb-2d3764639f28" containerID="9afbf4b49f4c4425205483986a2b91f30400dd4c4e4bb7100ebe4c354ba4e7fd" exitCode=0
Jul 07 15:08:40.112476 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:40.112026    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-d28bs" event={"ID":"ef55cae6-f66d-4a79-9cfb-2d3764639f28","Type":"ContainerDied","Data":"9afbf4b49f4c4425205483986a2b91f30400dd4c4e4bb7100ebe4c354ba4e7fd"}
Jul 07 15:08:40.112476 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:40.112033    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-d28bs"
Jul 07 15:08:40.112476 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:40.112077    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-d28bs" event={"ID":"ef55cae6-f66d-4a79-9cfb-2d3764639f28","Type":"ContainerDied","Data":"c1e31ce8e7fc0a54768b286c1328938370a6f0c11a245bb6e79eddac7855aba9"}
Jul 07 15:08:40.112476 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:40.112106    2567 scope.go:117] "RemoveContainer" containerID="9afbf4b49f4c4425205483986a2b91f30400dd4c4e4bb7100ebe4c354ba4e7fd"
Jul 07 15:08:40.120864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:40.120824    2567 scope.go:117] "RemoveContainer" containerID="9afbf4b49f4c4425205483986a2b91f30400dd4c4e4bb7100ebe4c354ba4e7fd"
Jul 07 15:08:40.121131 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:08:40.121112    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9afbf4b49f4c4425205483986a2b91f30400dd4c4e4bb7100ebe4c354ba4e7fd\": container with ID starting with 9afbf4b49f4c4425205483986a2b91f30400dd4c4e4bb7100ebe4c354ba4e7fd not found: ID does not exist" containerID="9afbf4b49f4c4425205483986a2b91f30400dd4c4e4bb7100ebe4c354ba4e7fd"
Jul 07 15:08:40.121178 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:40.121142    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9afbf4b49f4c4425205483986a2b91f30400dd4c4e4bb7100ebe4c354ba4e7fd"} err="failed to get container status \"9afbf4b49f4c4425205483986a2b91f30400dd4c4e4bb7100ebe4c354ba4e7fd\": rpc error: code = NotFound desc = could not find container \"9afbf4b49f4c4425205483986a2b91f30400dd4c4e4bb7100ebe4c354ba4e7fd\": container with ID starting with 9afbf4b49f4c4425205483986a2b91f30400dd4c4e4bb7100ebe4c354ba4e7fd not found: ID does not exist"
Jul 07 15:08:40.149996 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:40.149963    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-d28bs"]
Jul 07 15:08:40.151475 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:40.151456    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-d28bs"]
Jul 07 15:08:40.777447 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:40.777414    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef55cae6-f66d-4a79-9cfb-2d3764639f28" path="/var/lib/kubelet/pods/ef55cae6-f66d-4a79-9cfb-2d3764639f28/volumes"
Jul 07 15:08:43.615895 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.615863    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"]
Jul 07 15:08:43.616414 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.616255    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef55cae6-f66d-4a79-9cfb-2d3764639f28" containerName="manager"
Jul 07 15:08:43.616414 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.616268    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef55cae6-f66d-4a79-9cfb-2d3764639f28" containerName="manager"
Jul 07 15:08:43.616414 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.616358    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef55cae6-f66d-4a79-9cfb-2d3764639f28" containerName="manager"
Jul 07 15:08:43.619332 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.619312    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:43.622235 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.622211    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 07 15:08:43.622366 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.622211    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-dgmhq\""
Jul 07 15:08:43.636320 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.636298    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"]
Jul 07 15:08:43.758685 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.758645    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/497c7b71-8fa8-44b0-b75c-60f11cc9bd6c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-q9trj\" (UID: \"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:43.758885 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.758692    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l9hf\" (UniqueName: \"kubernetes.io/projected/497c7b71-8fa8-44b0-b75c-60f11cc9bd6c-kube-api-access-8l9hf\") pod \"maas-default-gateway-openshift-default-5896d4dd49-q9trj\" (UID: \"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:43.758885 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.758806    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/497c7b71-8fa8-44b0-b75c-60f11cc9bd6c-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-q9trj\" (UID: \"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:43.758971 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.758883    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/497c7b71-8fa8-44b0-b75c-60f11cc9bd6c-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-q9trj\" (UID: \"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:43.758971 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.758917    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/497c7b71-8fa8-44b0-b75c-60f11cc9bd6c-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-q9trj\" (UID: \"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:43.758971 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.758947    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/497c7b71-8fa8-44b0-b75c-60f11cc9bd6c-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-q9trj\" (UID: \"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:43.759064 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.758987    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/497c7b71-8fa8-44b0-b75c-60f11cc9bd6c-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-q9trj\" (UID: \"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:43.759064 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.759046    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/497c7b71-8fa8-44b0-b75c-60f11cc9bd6c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-q9trj\" (UID: \"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:43.759130 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.759074    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/497c7b71-8fa8-44b0-b75c-60f11cc9bd6c-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-q9trj\" (UID: \"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:43.860201 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.860169    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/497c7b71-8fa8-44b0-b75c-60f11cc9bd6c-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-q9trj\" (UID: \"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:43.860388 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.860216    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/497c7b71-8fa8-44b0-b75c-60f11cc9bd6c-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-q9trj\" (UID: \"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:43.860388 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.860341    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/497c7b71-8fa8-44b0-b75c-60f11cc9bd6c-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-q9trj\" (UID: \"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:43.860522 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.860405    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/497c7b71-8fa8-44b0-b75c-60f11cc9bd6c-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-q9trj\" (UID: \"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:43.860522 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.860446    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/497c7b71-8fa8-44b0-b75c-60f11cc9bd6c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-q9trj\" (UID: \"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:43.860522 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.860476    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/497c7b71-8fa8-44b0-b75c-60f11cc9bd6c-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-q9trj\" (UID: \"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:43.860670 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.860536    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/497c7b71-8fa8-44b0-b75c-60f11cc9bd6c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-q9trj\" (UID: \"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:43.860670 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.860580    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l9hf\" (UniqueName: \"kubernetes.io/projected/497c7b71-8fa8-44b0-b75c-60f11cc9bd6c-kube-api-access-8l9hf\") pod \"maas-default-gateway-openshift-default-5896d4dd49-q9trj\" (UID: \"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:43.860670 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.860589    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/497c7b71-8fa8-44b0-b75c-60f11cc9bd6c-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-q9trj\" (UID: \"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:43.860817 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.860671    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/497c7b71-8fa8-44b0-b75c-60f11cc9bd6c-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-q9trj\" (UID: \"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:43.860911 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.860873    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/497c7b71-8fa8-44b0-b75c-60f11cc9bd6c-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-q9trj\" (UID: \"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:43.861064 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.861040    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/497c7b71-8fa8-44b0-b75c-60f11cc9bd6c-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-q9trj\" (UID: \"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:43.861167 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.861084    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/497c7b71-8fa8-44b0-b75c-60f11cc9bd6c-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-q9trj\" (UID: \"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:43.861425 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.861401    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/497c7b71-8fa8-44b0-b75c-60f11cc9bd6c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-q9trj\" (UID: \"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:43.862725 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.862700    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/497c7b71-8fa8-44b0-b75c-60f11cc9bd6c-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-q9trj\" (UID: \"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:43.863137 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.863122    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/497c7b71-8fa8-44b0-b75c-60f11cc9bd6c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-q9trj\" (UID: \"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:43.869256 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.869199    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/497c7b71-8fa8-44b0-b75c-60f11cc9bd6c-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-q9trj\" (UID: \"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:43.869434 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.869417    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l9hf\" (UniqueName: \"kubernetes.io/projected/497c7b71-8fa8-44b0-b75c-60f11cc9bd6c-kube-api-access-8l9hf\") pod \"maas-default-gateway-openshift-default-5896d4dd49-q9trj\" (UID: \"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:43.930918 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:43.930883    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:44.082094 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:44.079199    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"]
Jul 07 15:08:44.132335 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:44.132249    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj" event={"ID":"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c","Type":"ContainerStarted","Data":"8052ec353e0bc4e4cf92aae76dd1889a33a0beaeb38ec9ec8cd60b6dd7d612ca"}
Jul 07 15:08:46.907232 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:46.907188    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:08:46.907477 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:46.907275    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:08:46.907477 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:46.907305    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:08:47.150060 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:47.150025    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj" event={"ID":"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c","Type":"ContainerStarted","Data":"ff431a11a520461fa09b63cea27d440558281b8f53e7ad4aab85c60bc93fe160"}
Jul 07 15:08:47.195220 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:47.195112    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj" podStartSLOduration=1.3724702469999999 podStartE2EDuration="4.195094552s" totalImagesPullingTime="2.822624305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:08:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:08:47.194072986 +0000 UTC m=+494.990609739" watchObservedRunningTime="2026-07-07 15:08:47.195094552 +0000 UTC m=+494.991631279"
Jul 07 15:08:47.931907 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:47.931869    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:47.936787 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:47.936756    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:48.154865 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:48.154808    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:48.155957 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:48.155934    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:08:58.599597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:58.599557    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-q5rzb"]
Jul 07 15:08:58.602685 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:58.602662    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-q5rzb"
Jul 07 15:08:58.606006 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:58.605987    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-29wfl\""
Jul 07 15:08:58.606262 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:58.606248    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 07 15:08:58.611024 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:58.611002    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-q5rzb"]
Jul 07 15:08:58.688557 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:58.688529    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/745f89f8-7213-4630-a2bb-b6378cc2a6d4-config-file\") pod \"limitador-limitador-64c94fd64f-q5rzb\" (UID: \"745f89f8-7213-4630-a2bb-b6378cc2a6d4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-q5rzb"
Jul 07 15:08:58.688725 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:58.688585    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lp272\" (UniqueName: \"kubernetes.io/projected/745f89f8-7213-4630-a2bb-b6378cc2a6d4-kube-api-access-lp272\") pod \"limitador-limitador-64c94fd64f-q5rzb\" (UID: \"745f89f8-7213-4630-a2bb-b6378cc2a6d4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-q5rzb"
Jul 07 15:08:58.693736 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:58.693707    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-q5rzb"]
Jul 07 15:08:58.789359 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:58.789334    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/745f89f8-7213-4630-a2bb-b6378cc2a6d4-config-file\") pod \"limitador-limitador-64c94fd64f-q5rzb\" (UID: \"745f89f8-7213-4630-a2bb-b6378cc2a6d4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-q5rzb"
Jul 07 15:08:58.789524 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:58.789378    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lp272\" (UniqueName: \"kubernetes.io/projected/745f89f8-7213-4630-a2bb-b6378cc2a6d4-kube-api-access-lp272\") pod \"limitador-limitador-64c94fd64f-q5rzb\" (UID: \"745f89f8-7213-4630-a2bb-b6378cc2a6d4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-q5rzb"
Jul 07 15:08:58.790014 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:58.789994    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/745f89f8-7213-4630-a2bb-b6378cc2a6d4-config-file\") pod \"limitador-limitador-64c94fd64f-q5rzb\" (UID: \"745f89f8-7213-4630-a2bb-b6378cc2a6d4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-q5rzb"
Jul 07 15:08:58.799136 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:58.799103    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lp272\" (UniqueName: \"kubernetes.io/projected/745f89f8-7213-4630-a2bb-b6378cc2a6d4-kube-api-access-lp272\") pod \"limitador-limitador-64c94fd64f-q5rzb\" (UID: \"745f89f8-7213-4630-a2bb-b6378cc2a6d4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-q5rzb"
Jul 07 15:08:58.913816 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:58.913721    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-q5rzb"
Jul 07 15:08:59.056237 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:59.056206    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-q5rzb"]
Jul 07 15:08:59.058811 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:08:59.058780    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod745f89f8_7213_4630_a2bb_b6378cc2a6d4.slice/crio-85f30813a7a4cc948b568754a59132431ca506409976506375378d655d34359c WatchSource:0}: Error finding container 85f30813a7a4cc948b568754a59132431ca506409976506375378d655d34359c: Status 404 returned error can't find the container with id 85f30813a7a4cc948b568754a59132431ca506409976506375378d655d34359c
Jul 07 15:08:59.214087 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:59.214005    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-q5rzb" event={"ID":"745f89f8-7213-4630-a2bb-b6378cc2a6d4","Type":"ContainerStarted","Data":"85f30813a7a4cc948b568754a59132431ca506409976506375378d655d34359c"}
Jul 07 15:08:59.428052 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:59.428019    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-n9cg8"]
Jul 07 15:08:59.431072 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:59.431040    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-n9cg8"
Jul 07 15:08:59.436028 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:59.436007    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-66xr8\""
Jul 07 15:08:59.443465 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:59.443435    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-n9cg8"]
Jul 07 15:08:59.594293 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:59.594255    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vltzj\" (UniqueName: \"kubernetes.io/projected/f80e8f3a-6da6-4323-b22a-23db94fe5c18-kube-api-access-vltzj\") pod \"authorino-58cd955675-n9cg8\" (UID: \"f80e8f3a-6da6-4323-b22a-23db94fe5c18\") " pod="kuadrant-system/authorino-58cd955675-n9cg8"
Jul 07 15:08:59.694746 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:59.694709    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vltzj\" (UniqueName: \"kubernetes.io/projected/f80e8f3a-6da6-4323-b22a-23db94fe5c18-kube-api-access-vltzj\") pod \"authorino-58cd955675-n9cg8\" (UID: \"f80e8f3a-6da6-4323-b22a-23db94fe5c18\") " pod="kuadrant-system/authorino-58cd955675-n9cg8"
Jul 07 15:08:59.704290 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:59.704263    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vltzj\" (UniqueName: \"kubernetes.io/projected/f80e8f3a-6da6-4323-b22a-23db94fe5c18-kube-api-access-vltzj\") pod \"authorino-58cd955675-n9cg8\" (UID: \"f80e8f3a-6da6-4323-b22a-23db94fe5c18\") " pod="kuadrant-system/authorino-58cd955675-n9cg8"
Jul 07 15:08:59.740541 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:59.740511    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-n9cg8"
Jul 07 15:08:59.882918 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:08:59.882893    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-n9cg8"]
Jul 07 15:08:59.884277 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:08:59.884249    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf80e8f3a_6da6_4323_b22a_23db94fe5c18.slice/crio-06457531ce6f63eb291ecc97aca53ebfc2642c4a42e8dcae24874a8ffefad373 WatchSource:0}: Error finding container 06457531ce6f63eb291ecc97aca53ebfc2642c4a42e8dcae24874a8ffefad373: Status 404 returned error can't find the container with id 06457531ce6f63eb291ecc97aca53ebfc2642c4a42e8dcae24874a8ffefad373
Jul 07 15:09:00.222537 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:00.222450    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-n9cg8" event={"ID":"f80e8f3a-6da6-4323-b22a-23db94fe5c18","Type":"ContainerStarted","Data":"06457531ce6f63eb291ecc97aca53ebfc2642c4a42e8dcae24874a8ffefad373"}
Jul 07 15:09:04.244881 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:04.244821    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-n9cg8" event={"ID":"f80e8f3a-6da6-4323-b22a-23db94fe5c18","Type":"ContainerStarted","Data":"23346d01d77dcd87a06b88ad211765bf157550f951c0c07723fc255b3d829089"}
Jul 07 15:09:04.246318 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:04.246298    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-q5rzb" event={"ID":"745f89f8-7213-4630-a2bb-b6378cc2a6d4","Type":"ContainerStarted","Data":"e0022ddbf46a3d5039ddc83a98733ac394f32ffcca9bdd7f9b8923b8e6be15ad"}
Jul 07 15:09:04.246431 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:04.246418    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-q5rzb"
Jul 07 15:09:04.264925 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:04.264884    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-n9cg8" podStartSLOduration=1.104186292 podStartE2EDuration="5.264869706s" totalImagesPullingTime="4.160683414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:08:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:09:04.263071949 +0000 UTC m=+512.059608700" watchObservedRunningTime="2026-07-07 15:09:04.264869706 +0000 UTC m=+512.061406446"
Jul 07 15:09:04.288575 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:04.288467    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-q5rzb" podStartSLOduration=1.291455923 podStartE2EDuration="6.288448176s" totalImagesPullingTime="4.996992253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:08:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:09:04.286351972 +0000 UTC m=+512.082888732" watchObservedRunningTime="2026-07-07 15:09:04.288448176 +0000 UTC m=+512.084984906"
Jul 07 15:09:05.143504 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:05.143470    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-n9cg8"]
Jul 07 15:09:06.257198 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:06.257136    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-n9cg8" podUID="f80e8f3a-6da6-4323-b22a-23db94fe5c18" containerName="authorino" containerID="cri-o://23346d01d77dcd87a06b88ad211765bf157550f951c0c07723fc255b3d829089" gracePeriod=30
Jul 07 15:09:06.516392 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:06.516334    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-n9cg8"
Jul 07 15:09:06.661986 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:06.661954    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vltzj\" (UniqueName: \"kubernetes.io/projected/f80e8f3a-6da6-4323-b22a-23db94fe5c18-kube-api-access-vltzj\") pod \"f80e8f3a-6da6-4323-b22a-23db94fe5c18\" (UID: \"f80e8f3a-6da6-4323-b22a-23db94fe5c18\") "
Jul 07 15:09:06.664233 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:06.664196    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f80e8f3a-6da6-4323-b22a-23db94fe5c18-kube-api-access-vltzj" (OuterVolumeSpecName: "kube-api-access-vltzj") pod "f80e8f3a-6da6-4323-b22a-23db94fe5c18" (UID: "f80e8f3a-6da6-4323-b22a-23db94fe5c18"). InnerVolumeSpecName "kube-api-access-vltzj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:09:06.762810 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:06.762776    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vltzj\" (UniqueName: \"kubernetes.io/projected/f80e8f3a-6da6-4323-b22a-23db94fe5c18-kube-api-access-vltzj\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:09:07.263271 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:07.263234    2567 generic.go:356] "Generic (PLEG): container finished" podID="f80e8f3a-6da6-4323-b22a-23db94fe5c18" containerID="23346d01d77dcd87a06b88ad211765bf157550f951c0c07723fc255b3d829089" exitCode=0
Jul 07 15:09:07.263271 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:07.263269    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-n9cg8"
Jul 07 15:09:07.263271 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:07.263275    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-n9cg8" event={"ID":"f80e8f3a-6da6-4323-b22a-23db94fe5c18","Type":"ContainerDied","Data":"23346d01d77dcd87a06b88ad211765bf157550f951c0c07723fc255b3d829089"}
Jul 07 15:09:07.263814 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:07.263302    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-n9cg8" event={"ID":"f80e8f3a-6da6-4323-b22a-23db94fe5c18","Type":"ContainerDied","Data":"06457531ce6f63eb291ecc97aca53ebfc2642c4a42e8dcae24874a8ffefad373"}
Jul 07 15:09:07.263814 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:07.263319    2567 scope.go:117] "RemoveContainer" containerID="23346d01d77dcd87a06b88ad211765bf157550f951c0c07723fc255b3d829089"
Jul 07 15:09:07.271707 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:07.271690    2567 scope.go:117] "RemoveContainer" containerID="23346d01d77dcd87a06b88ad211765bf157550f951c0c07723fc255b3d829089"
Jul 07 15:09:07.271998 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:09:07.271979    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23346d01d77dcd87a06b88ad211765bf157550f951c0c07723fc255b3d829089\": container with ID starting with 23346d01d77dcd87a06b88ad211765bf157550f951c0c07723fc255b3d829089 not found: ID does not exist" containerID="23346d01d77dcd87a06b88ad211765bf157550f951c0c07723fc255b3d829089"
Jul 07 15:09:07.272068 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:07.272010    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23346d01d77dcd87a06b88ad211765bf157550f951c0c07723fc255b3d829089"} err="failed to get container status \"23346d01d77dcd87a06b88ad211765bf157550f951c0c07723fc255b3d829089\": rpc error: code = NotFound desc = could not find container \"23346d01d77dcd87a06b88ad211765bf157550f951c0c07723fc255b3d829089\": container with ID starting with 23346d01d77dcd87a06b88ad211765bf157550f951c0c07723fc255b3d829089 not found: ID does not exist"
Jul 07 15:09:07.281200 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:07.281176    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-n9cg8"]
Jul 07 15:09:07.285804 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:07.285783    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-n9cg8"]
Jul 07 15:09:08.782543 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:08.782508    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f80e8f3a-6da6-4323-b22a-23db94fe5c18" path="/var/lib/kubelet/pods/f80e8f3a-6da6-4323-b22a-23db94fe5c18/volumes"
Jul 07 15:09:15.254329 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:15.254296    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-q5rzb"
Jul 07 15:09:16.117381 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:16.117348    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-q5rzb"]
Jul 07 15:09:16.117623 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:16.117590    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-q5rzb" podUID="745f89f8-7213-4630-a2bb-b6378cc2a6d4" containerName="limitador" containerID="cri-o://e0022ddbf46a3d5039ddc83a98733ac394f32ffcca9bdd7f9b8923b8e6be15ad" gracePeriod=30
Jul 07 15:09:16.673128 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:16.673104    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-q5rzb"
Jul 07 15:09:16.851241 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:16.851211    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lp272\" (UniqueName: \"kubernetes.io/projected/745f89f8-7213-4630-a2bb-b6378cc2a6d4-kube-api-access-lp272\") pod \"745f89f8-7213-4630-a2bb-b6378cc2a6d4\" (UID: \"745f89f8-7213-4630-a2bb-b6378cc2a6d4\") "
Jul 07 15:09:16.851432 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:16.851297    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/745f89f8-7213-4630-a2bb-b6378cc2a6d4-config-file\") pod \"745f89f8-7213-4630-a2bb-b6378cc2a6d4\" (UID: \"745f89f8-7213-4630-a2bb-b6378cc2a6d4\") "
Jul 07 15:09:16.851659 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:16.851630    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/745f89f8-7213-4630-a2bb-b6378cc2a6d4-config-file" (OuterVolumeSpecName: "config-file") pod "745f89f8-7213-4630-a2bb-b6378cc2a6d4" (UID: "745f89f8-7213-4630-a2bb-b6378cc2a6d4"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:09:16.853440 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:16.853416    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/745f89f8-7213-4630-a2bb-b6378cc2a6d4-kube-api-access-lp272" (OuterVolumeSpecName: "kube-api-access-lp272") pod "745f89f8-7213-4630-a2bb-b6378cc2a6d4" (UID: "745f89f8-7213-4630-a2bb-b6378cc2a6d4"). InnerVolumeSpecName "kube-api-access-lp272". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:09:16.952341 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:16.952301    2567 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/745f89f8-7213-4630-a2bb-b6378cc2a6d4-config-file\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:09:16.952341 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:16.952334    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lp272\" (UniqueName: \"kubernetes.io/projected/745f89f8-7213-4630-a2bb-b6378cc2a6d4-kube-api-access-lp272\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:09:17.314800 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:17.314763    2567 generic.go:356] "Generic (PLEG): container finished" podID="745f89f8-7213-4630-a2bb-b6378cc2a6d4" containerID="e0022ddbf46a3d5039ddc83a98733ac394f32ffcca9bdd7f9b8923b8e6be15ad" exitCode=0
Jul 07 15:09:17.315000 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:17.314811    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-q5rzb"
Jul 07 15:09:17.315000 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:17.314853    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-q5rzb" event={"ID":"745f89f8-7213-4630-a2bb-b6378cc2a6d4","Type":"ContainerDied","Data":"e0022ddbf46a3d5039ddc83a98733ac394f32ffcca9bdd7f9b8923b8e6be15ad"}
Jul 07 15:09:17.315000 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:17.314905    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-q5rzb" event={"ID":"745f89f8-7213-4630-a2bb-b6378cc2a6d4","Type":"ContainerDied","Data":"85f30813a7a4cc948b568754a59132431ca506409976506375378d655d34359c"}
Jul 07 15:09:17.315000 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:17.314927    2567 scope.go:117] "RemoveContainer" containerID="e0022ddbf46a3d5039ddc83a98733ac394f32ffcca9bdd7f9b8923b8e6be15ad"
Jul 07 15:09:17.324086 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:17.324067    2567 scope.go:117] "RemoveContainer" containerID="e0022ddbf46a3d5039ddc83a98733ac394f32ffcca9bdd7f9b8923b8e6be15ad"
Jul 07 15:09:17.324348 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:09:17.324324    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0022ddbf46a3d5039ddc83a98733ac394f32ffcca9bdd7f9b8923b8e6be15ad\": container with ID starting with e0022ddbf46a3d5039ddc83a98733ac394f32ffcca9bdd7f9b8923b8e6be15ad not found: ID does not exist" containerID="e0022ddbf46a3d5039ddc83a98733ac394f32ffcca9bdd7f9b8923b8e6be15ad"
Jul 07 15:09:17.324431 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:17.324354    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0022ddbf46a3d5039ddc83a98733ac394f32ffcca9bdd7f9b8923b8e6be15ad"} err="failed to get container status \"e0022ddbf46a3d5039ddc83a98733ac394f32ffcca9bdd7f9b8923b8e6be15ad\": rpc error: code = NotFound desc = could not find container \"e0022ddbf46a3d5039ddc83a98733ac394f32ffcca9bdd7f9b8923b8e6be15ad\": container with ID starting with e0022ddbf46a3d5039ddc83a98733ac394f32ffcca9bdd7f9b8923b8e6be15ad not found: ID does not exist"
Jul 07 15:09:17.338957 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:17.338933    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-q5rzb"]
Jul 07 15:09:17.347163 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:17.347137    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-q5rzb"]
Jul 07 15:09:18.777589 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:18.777556    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="745f89f8-7213-4630-a2bb-b6378cc2a6d4" path="/var/lib/kubelet/pods/745f89f8-7213-4630-a2bb-b6378cc2a6d4/volumes"
Jul 07 15:09:20.222124 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:20.222087    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-cvwzp"]
Jul 07 15:09:20.222636 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:20.222573    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f80e8f3a-6da6-4323-b22a-23db94fe5c18" containerName="authorino"
Jul 07 15:09:20.222636 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:20.222592    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f80e8f3a-6da6-4323-b22a-23db94fe5c18" containerName="authorino"
Jul 07 15:09:20.222636 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:20.222604    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="745f89f8-7213-4630-a2bb-b6378cc2a6d4" containerName="limitador"
Jul 07 15:09:20.222636 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:20.222611    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="745f89f8-7213-4630-a2bb-b6378cc2a6d4" containerName="limitador"
Jul 07 15:09:20.222900 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:20.222698    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="745f89f8-7213-4630-a2bb-b6378cc2a6d4" containerName="limitador"
Jul 07 15:09:20.222900 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:20.222714    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f80e8f3a-6da6-4323-b22a-23db94fe5c18" containerName="authorino"
Jul 07 15:09:20.227864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:20.227818    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-cvwzp"
Jul 07 15:09:20.230727 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:20.230695    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\""
Jul 07 15:09:20.230844 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:20.230754    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-jdc48\""
Jul 07 15:09:20.234568 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:20.234547    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-cvwzp"]
Jul 07 15:09:20.278938 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:20.278909    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llxwh\" (UniqueName: \"kubernetes.io/projected/bfc2459b-ebe8-4a93-8978-87a7b524b49d-kube-api-access-llxwh\") pod \"postgres-69fb4594cf-cvwzp\" (UID: \"bfc2459b-ebe8-4a93-8978-87a7b524b49d\") " pod="opendatahub/postgres-69fb4594cf-cvwzp"
Jul 07 15:09:20.279102 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:20.278966    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/bfc2459b-ebe8-4a93-8978-87a7b524b49d-data\") pod \"postgres-69fb4594cf-cvwzp\" (UID: \"bfc2459b-ebe8-4a93-8978-87a7b524b49d\") " pod="opendatahub/postgres-69fb4594cf-cvwzp"
Jul 07 15:09:20.379608 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:20.379568    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llxwh\" (UniqueName: \"kubernetes.io/projected/bfc2459b-ebe8-4a93-8978-87a7b524b49d-kube-api-access-llxwh\") pod \"postgres-69fb4594cf-cvwzp\" (UID: \"bfc2459b-ebe8-4a93-8978-87a7b524b49d\") " pod="opendatahub/postgres-69fb4594cf-cvwzp"
Jul 07 15:09:20.379787 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:20.379644    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/bfc2459b-ebe8-4a93-8978-87a7b524b49d-data\") pod \"postgres-69fb4594cf-cvwzp\" (UID: \"bfc2459b-ebe8-4a93-8978-87a7b524b49d\") " pod="opendatahub/postgres-69fb4594cf-cvwzp"
Jul 07 15:09:20.380058 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:20.380038    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/bfc2459b-ebe8-4a93-8978-87a7b524b49d-data\") pod \"postgres-69fb4594cf-cvwzp\" (UID: \"bfc2459b-ebe8-4a93-8978-87a7b524b49d\") " pod="opendatahub/postgres-69fb4594cf-cvwzp"
Jul 07 15:09:20.390925 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:20.390901    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-llxwh\" (UniqueName: \"kubernetes.io/projected/bfc2459b-ebe8-4a93-8978-87a7b524b49d-kube-api-access-llxwh\") pod \"postgres-69fb4594cf-cvwzp\" (UID: \"bfc2459b-ebe8-4a93-8978-87a7b524b49d\") " pod="opendatahub/postgres-69fb4594cf-cvwzp"
Jul 07 15:09:20.539995 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:20.539896    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-cvwzp"
Jul 07 15:09:20.678232 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:20.678202    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-cvwzp"]
Jul 07 15:09:20.681238 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:09:20.681209    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfc2459b_ebe8_4a93_8978_87a7b524b49d.slice/crio-d8a71c18901a3a061c8612304f5c48257a8dfb9a42e42997340668cc58ad9306 WatchSource:0}: Error finding container d8a71c18901a3a061c8612304f5c48257a8dfb9a42e42997340668cc58ad9306: Status 404 returned error can't find the container with id d8a71c18901a3a061c8612304f5c48257a8dfb9a42e42997340668cc58ad9306
Jul 07 15:09:21.342478 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:21.342435    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-cvwzp" event={"ID":"bfc2459b-ebe8-4a93-8978-87a7b524b49d","Type":"ContainerStarted","Data":"d8a71c18901a3a061c8612304f5c48257a8dfb9a42e42997340668cc58ad9306"}
Jul 07 15:09:26.372108 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:26.372065    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-cvwzp" event={"ID":"bfc2459b-ebe8-4a93-8978-87a7b524b49d","Type":"ContainerStarted","Data":"566256c6ea02125663c3b32acd63056326f65c64c6885194107c1679413a4cda"}
Jul 07 15:09:26.372494 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:26.372195    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-cvwzp"
Jul 07 15:09:26.390280 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:26.390229    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-cvwzp" podStartSLOduration=1.420029972 podStartE2EDuration="6.390213049s" totalImagesPullingTime="4.970183077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:09:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:09:26.389165933 +0000 UTC m=+534.185702675" watchObservedRunningTime="2026-07-07 15:09:26.390213049 +0000 UTC m=+534.186749794"
Jul 07 15:09:32.409213 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:32.409186    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-cvwzp"
Jul 07 15:09:33.026047 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.025963    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-rxwjp"]
Jul 07 15:09:33.066605 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.066571    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-rxwjp"]
Jul 07 15:09:33.066789 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.066687    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-rxwjp"
Jul 07 15:09:33.078020 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.077998    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-66xr8\""
Jul 07 15:09:33.190820 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.190783    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqqnh\" (UniqueName: \"kubernetes.io/projected/5f991979-6b44-4947-a00d-6abe92586e2e-kube-api-access-vqqnh\") pod \"authorino-5d4b67c446-rxwjp\" (UID: \"5f991979-6b44-4947-a00d-6abe92586e2e\") " pod="kuadrant-system/authorino-5d4b67c446-rxwjp"
Jul 07 15:09:33.292054 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.291951    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqqnh\" (UniqueName: \"kubernetes.io/projected/5f991979-6b44-4947-a00d-6abe92586e2e-kube-api-access-vqqnh\") pod \"authorino-5d4b67c446-rxwjp\" (UID: \"5f991979-6b44-4947-a00d-6abe92586e2e\") " pod="kuadrant-system/authorino-5d4b67c446-rxwjp"
Jul 07 15:09:33.294021 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.293998    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-rxwjp"]
Jul 07 15:09:33.294232 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:09:33.294205    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-vqqnh], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-rxwjp" podUID="5f991979-6b44-4947-a00d-6abe92586e2e"
Jul 07 15:09:33.301505 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.301481    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqqnh\" (UniqueName: \"kubernetes.io/projected/5f991979-6b44-4947-a00d-6abe92586e2e-kube-api-access-vqqnh\") pod \"authorino-5d4b67c446-rxwjp\" (UID: \"5f991979-6b44-4947-a00d-6abe92586e2e\") " pod="kuadrant-system/authorino-5d4b67c446-rxwjp"
Jul 07 15:09:33.329026 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.328993    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7dfb677dd6-w4qpj"]
Jul 07 15:09:33.370679 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.370648    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7dfb677dd6-w4qpj"]
Jul 07 15:09:33.370883 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.370739    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7dfb677dd6-w4qpj"
Jul 07 15:09:33.406560 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.406527    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-rxwjp"
Jul 07 15:09:33.430313 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.430287    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-rxwjp"
Jul 07 15:09:33.493977 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.493950    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vqqnh\" (UniqueName: \"kubernetes.io/projected/5f991979-6b44-4947-a00d-6abe92586e2e-kube-api-access-vqqnh\") pod \"5f991979-6b44-4947-a00d-6abe92586e2e\" (UID: \"5f991979-6b44-4947-a00d-6abe92586e2e\") "
Jul 07 15:09:33.494151 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.494125    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwtck\" (UniqueName: \"kubernetes.io/projected/f9968734-3c1c-48ae-bc41-7c8a883137a7-kube-api-access-nwtck\") pod \"authorino-7dfb677dd6-w4qpj\" (UID: \"f9968734-3c1c-48ae-bc41-7c8a883137a7\") " pod="kuadrant-system/authorino-7dfb677dd6-w4qpj"
Jul 07 15:09:33.496208 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.496180    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f991979-6b44-4947-a00d-6abe92586e2e-kube-api-access-vqqnh" (OuterVolumeSpecName: "kube-api-access-vqqnh") pod "5f991979-6b44-4947-a00d-6abe92586e2e" (UID: "5f991979-6b44-4947-a00d-6abe92586e2e"). InnerVolumeSpecName "kube-api-access-vqqnh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:09:33.499521 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.499499    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7dfb677dd6-w4qpj"]
Jul 07 15:09:33.499742 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:09:33.499724    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-nwtck], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-7dfb677dd6-w4qpj" podUID="f9968734-3c1c-48ae-bc41-7c8a883137a7"
Jul 07 15:09:33.532961 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.532923    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5b586fb75d-2j8cw"]
Jul 07 15:09:33.554192 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.554160    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5b586fb75d-2j8cw"]
Jul 07 15:09:33.554365 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.554251    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b586fb75d-2j8cw"
Jul 07 15:09:33.558285 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.558264    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 07 15:09:33.595152 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.595117    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwtck\" (UniqueName: \"kubernetes.io/projected/f9968734-3c1c-48ae-bc41-7c8a883137a7-kube-api-access-nwtck\") pod \"authorino-7dfb677dd6-w4qpj\" (UID: \"f9968734-3c1c-48ae-bc41-7c8a883137a7\") " pod="kuadrant-system/authorino-7dfb677dd6-w4qpj"
Jul 07 15:09:33.595344 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.595176    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6b35acac-02b1-4c22-a6ee-6bc321d7d0b0-tls-cert\") pod \"authorino-5b586fb75d-2j8cw\" (UID: \"6b35acac-02b1-4c22-a6ee-6bc321d7d0b0\") " pod="kuadrant-system/authorino-5b586fb75d-2j8cw"
Jul 07 15:09:33.595344 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.595275    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkjrs\" (UniqueName: \"kubernetes.io/projected/6b35acac-02b1-4c22-a6ee-6bc321d7d0b0-kube-api-access-pkjrs\") pod \"authorino-5b586fb75d-2j8cw\" (UID: \"6b35acac-02b1-4c22-a6ee-6bc321d7d0b0\") " pod="kuadrant-system/authorino-5b586fb75d-2j8cw"
Jul 07 15:09:33.595430 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.595359    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqqnh\" (UniqueName: \"kubernetes.io/projected/5f991979-6b44-4947-a00d-6abe92586e2e-kube-api-access-vqqnh\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:09:33.605308 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.605286    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwtck\" (UniqueName: \"kubernetes.io/projected/f9968734-3c1c-48ae-bc41-7c8a883137a7-kube-api-access-nwtck\") pod \"authorino-7dfb677dd6-w4qpj\" (UID: \"f9968734-3c1c-48ae-bc41-7c8a883137a7\") " pod="kuadrant-system/authorino-7dfb677dd6-w4qpj"
Jul 07 15:09:33.696397 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.696363    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkjrs\" (UniqueName: \"kubernetes.io/projected/6b35acac-02b1-4c22-a6ee-6bc321d7d0b0-kube-api-access-pkjrs\") pod \"authorino-5b586fb75d-2j8cw\" (UID: \"6b35acac-02b1-4c22-a6ee-6bc321d7d0b0\") " pod="kuadrant-system/authorino-5b586fb75d-2j8cw"
Jul 07 15:09:33.696554 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.696431    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6b35acac-02b1-4c22-a6ee-6bc321d7d0b0-tls-cert\") pod \"authorino-5b586fb75d-2j8cw\" (UID: \"6b35acac-02b1-4c22-a6ee-6bc321d7d0b0\") " pod="kuadrant-system/authorino-5b586fb75d-2j8cw"
Jul 07 15:09:33.699003 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.698975    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6b35acac-02b1-4c22-a6ee-6bc321d7d0b0-tls-cert\") pod \"authorino-5b586fb75d-2j8cw\" (UID: \"6b35acac-02b1-4c22-a6ee-6bc321d7d0b0\") " pod="kuadrant-system/authorino-5b586fb75d-2j8cw"
Jul 07 15:09:33.706705 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.706675    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkjrs\" (UniqueName: \"kubernetes.io/projected/6b35acac-02b1-4c22-a6ee-6bc321d7d0b0-kube-api-access-pkjrs\") pod \"authorino-5b586fb75d-2j8cw\" (UID: \"6b35acac-02b1-4c22-a6ee-6bc321d7d0b0\") " pod="kuadrant-system/authorino-5b586fb75d-2j8cw"
Jul 07 15:09:33.864013 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:33.863939    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b586fb75d-2j8cw"
Jul 07 15:09:34.025505 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:34.025479    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5b586fb75d-2j8cw"]
Jul 07 15:09:34.027524 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:09:34.027483    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b35acac_02b1_4c22_a6ee_6bc321d7d0b0.slice/crio-898d9f71f31383333a440eab05b23b71a0e06e778d95746d08439c073665ff4b WatchSource:0}: Error finding container 898d9f71f31383333a440eab05b23b71a0e06e778d95746d08439c073665ff4b: Status 404 returned error can't find the container with id 898d9f71f31383333a440eab05b23b71a0e06e778d95746d08439c073665ff4b
Jul 07 15:09:34.412473 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:34.412438    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5b586fb75d-2j8cw" event={"ID":"6b35acac-02b1-4c22-a6ee-6bc321d7d0b0","Type":"ContainerStarted","Data":"898d9f71f31383333a440eab05b23b71a0e06e778d95746d08439c073665ff4b"}
Jul 07 15:09:34.412625 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:34.412495    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7dfb677dd6-w4qpj"
Jul 07 15:09:34.412625 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:34.412500    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-rxwjp"
Jul 07 15:09:34.445626 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:34.445594    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7dfb677dd6-w4qpj"
Jul 07 15:09:34.469905 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:34.469875    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-rxwjp"]
Jul 07 15:09:34.474148 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:34.474125    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-rxwjp"]
Jul 07 15:09:34.504514 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:34.504484    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nwtck\" (UniqueName: \"kubernetes.io/projected/f9968734-3c1c-48ae-bc41-7c8a883137a7-kube-api-access-nwtck\") pod \"f9968734-3c1c-48ae-bc41-7c8a883137a7\" (UID: \"f9968734-3c1c-48ae-bc41-7c8a883137a7\") "
Jul 07 15:09:34.506621 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:34.506597    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9968734-3c1c-48ae-bc41-7c8a883137a7-kube-api-access-nwtck" (OuterVolumeSpecName: "kube-api-access-nwtck") pod "f9968734-3c1c-48ae-bc41-7c8a883137a7" (UID: "f9968734-3c1c-48ae-bc41-7c8a883137a7"). InnerVolumeSpecName "kube-api-access-nwtck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:09:34.605433 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:34.605399    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwtck\" (UniqueName: \"kubernetes.io/projected/f9968734-3c1c-48ae-bc41-7c8a883137a7-kube-api-access-nwtck\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:09:34.777642 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:34.777564    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f991979-6b44-4947-a00d-6abe92586e2e" path="/var/lib/kubelet/pods/5f991979-6b44-4947-a00d-6abe92586e2e/volumes"
Jul 07 15:09:35.418421 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:35.418377    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5b586fb75d-2j8cw" event={"ID":"6b35acac-02b1-4c22-a6ee-6bc321d7d0b0","Type":"ContainerStarted","Data":"1da112a5199df80e33f14009a4655a2b3ff93e919ad311cfdcba1c7e10ddc559"}
Jul 07 15:09:35.418421 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:35.418407    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7dfb677dd6-w4qpj"
Jul 07 15:09:35.441518 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:35.441468    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5b586fb75d-2j8cw" podStartSLOduration=1.448931234 podStartE2EDuration="2.441452525s" totalImagesPullingTime="992.521291ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:09:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:09:35.441016116 +0000 UTC m=+543.237552843" watchObservedRunningTime="2026-07-07 15:09:35.441452525 +0000 UTC m=+543.237989252"
Jul 07 15:09:35.470222 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:35.470171    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7dfb677dd6-w4qpj"]
Jul 07 15:09:35.472146 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:35.472121    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7dfb677dd6-w4qpj"]
Jul 07 15:09:36.777897 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:36.777865    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9968734-3c1c-48ae-bc41-7c8a883137a7" path="/var/lib/kubelet/pods/f9968734-3c1c-48ae-bc41-7c8a883137a7/volumes"
Jul 07 15:09:39.983052 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:39.983017    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-57b998dcff-nkdvf"]
Jul 07 15:09:39.986022 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:39.986002    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57b998dcff-nkdvf"
Jul 07 15:09:39.988886 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:39.988851    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 07 15:09:39.988886 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:39.988882    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 07 15:09:39.989061 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:39.988931    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dfpjj\""
Jul 07 15:09:39.999565 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:39.999537    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57b998dcff-nkdvf"]
Jul 07 15:09:40.065749 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:40.065714    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a67bcb42-318f-43a3-b3d5-70dba7ab4915-webhook-cert\") pod \"maas-controller-57b998dcff-nkdvf\" (UID: \"a67bcb42-318f-43a3-b3d5-70dba7ab4915\") " pod="opendatahub/maas-controller-57b998dcff-nkdvf"
Jul 07 15:09:40.065970 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:40.065761    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9pzs\" (UniqueName: \"kubernetes.io/projected/a67bcb42-318f-43a3-b3d5-70dba7ab4915-kube-api-access-p9pzs\") pod \"maas-controller-57b998dcff-nkdvf\" (UID: \"a67bcb42-318f-43a3-b3d5-70dba7ab4915\") " pod="opendatahub/maas-controller-57b998dcff-nkdvf"
Jul 07 15:09:40.161668 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:40.161633    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5b477bdf49-dln4b"]
Jul 07 15:09:40.164360 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:40.164340    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b477bdf49-dln4b"
Jul 07 15:09:40.166483 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:40.166460    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9pzs\" (UniqueName: \"kubernetes.io/projected/a67bcb42-318f-43a3-b3d5-70dba7ab4915-kube-api-access-p9pzs\") pod \"maas-controller-57b998dcff-nkdvf\" (UID: \"a67bcb42-318f-43a3-b3d5-70dba7ab4915\") " pod="opendatahub/maas-controller-57b998dcff-nkdvf"
Jul 07 15:09:40.166605 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:40.166560    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a67bcb42-318f-43a3-b3d5-70dba7ab4915-webhook-cert\") pod \"maas-controller-57b998dcff-nkdvf\" (UID: \"a67bcb42-318f-43a3-b3d5-70dba7ab4915\") " pod="opendatahub/maas-controller-57b998dcff-nkdvf"
Jul 07 15:09:40.169066 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:40.169046    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a67bcb42-318f-43a3-b3d5-70dba7ab4915-webhook-cert\") pod \"maas-controller-57b998dcff-nkdvf\" (UID: \"a67bcb42-318f-43a3-b3d5-70dba7ab4915\") " pod="opendatahub/maas-controller-57b998dcff-nkdvf"
Jul 07 15:09:40.175905 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:40.175881    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b477bdf49-dln4b"]
Jul 07 15:09:40.178216 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:40.178192    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9pzs\" (UniqueName: \"kubernetes.io/projected/a67bcb42-318f-43a3-b3d5-70dba7ab4915-kube-api-access-p9pzs\") pod \"maas-controller-57b998dcff-nkdvf\" (UID: \"a67bcb42-318f-43a3-b3d5-70dba7ab4915\") " pod="opendatahub/maas-controller-57b998dcff-nkdvf"
Jul 07 15:09:40.267137 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:40.267039    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84e8f9e9-dd85-4b04-8594-1521b9a71765-webhook-cert\") pod \"maas-controller-5b477bdf49-dln4b\" (UID: \"84e8f9e9-dd85-4b04-8594-1521b9a71765\") " pod="opendatahub/maas-controller-5b477bdf49-dln4b"
Jul 07 15:09:40.267137 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:40.267088    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dlvr\" (UniqueName: \"kubernetes.io/projected/84e8f9e9-dd85-4b04-8594-1521b9a71765-kube-api-access-5dlvr\") pod \"maas-controller-5b477bdf49-dln4b\" (UID: \"84e8f9e9-dd85-4b04-8594-1521b9a71765\") " pod="opendatahub/maas-controller-5b477bdf49-dln4b"
Jul 07 15:09:40.296705 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:40.296671    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57b998dcff-nkdvf"
Jul 07 15:09:40.368124 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:40.368095    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84e8f9e9-dd85-4b04-8594-1521b9a71765-webhook-cert\") pod \"maas-controller-5b477bdf49-dln4b\" (UID: \"84e8f9e9-dd85-4b04-8594-1521b9a71765\") " pod="opendatahub/maas-controller-5b477bdf49-dln4b"
Jul 07 15:09:40.368277 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:40.368129    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dlvr\" (UniqueName: \"kubernetes.io/projected/84e8f9e9-dd85-4b04-8594-1521b9a71765-kube-api-access-5dlvr\") pod \"maas-controller-5b477bdf49-dln4b\" (UID: \"84e8f9e9-dd85-4b04-8594-1521b9a71765\") " pod="opendatahub/maas-controller-5b477bdf49-dln4b"
Jul 07 15:09:40.370773 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:40.370746    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84e8f9e9-dd85-4b04-8594-1521b9a71765-webhook-cert\") pod \"maas-controller-5b477bdf49-dln4b\" (UID: \"84e8f9e9-dd85-4b04-8594-1521b9a71765\") " pod="opendatahub/maas-controller-5b477bdf49-dln4b"
Jul 07 15:09:40.379440 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:40.379417    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dlvr\" (UniqueName: \"kubernetes.io/projected/84e8f9e9-dd85-4b04-8594-1521b9a71765-kube-api-access-5dlvr\") pod \"maas-controller-5b477bdf49-dln4b\" (UID: \"84e8f9e9-dd85-4b04-8594-1521b9a71765\") " pod="opendatahub/maas-controller-5b477bdf49-dln4b"
Jul 07 15:09:40.434664 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:40.434624    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57b998dcff-nkdvf"]
Jul 07 15:09:40.436874 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:09:40.436823    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda67bcb42_318f_43a3_b3d5_70dba7ab4915.slice/crio-7b7da79ba14c355301fb3bff246a6b6f4e468be43359718dc85e090836d7970b WatchSource:0}: Error finding container 7b7da79ba14c355301fb3bff246a6b6f4e468be43359718dc85e090836d7970b: Status 404 returned error can't find the container with id 7b7da79ba14c355301fb3bff246a6b6f4e468be43359718dc85e090836d7970b
Jul 07 15:09:40.448907 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:40.448871    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57b998dcff-nkdvf" event={"ID":"a67bcb42-318f-43a3-b3d5-70dba7ab4915","Type":"ContainerStarted","Data":"7b7da79ba14c355301fb3bff246a6b6f4e468be43359718dc85e090836d7970b"}
Jul 07 15:09:40.496382 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:40.496329    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b477bdf49-dln4b"
Jul 07 15:09:40.635115 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:40.635091    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b477bdf49-dln4b"]
Jul 07 15:09:40.636863 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:09:40.636816    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84e8f9e9_dd85_4b04_8594_1521b9a71765.slice/crio-25472cafa14d86c0a14cf331da39bbdf0bf831b4917a63bff8ccead5037b4a1b WatchSource:0}: Error finding container 25472cafa14d86c0a14cf331da39bbdf0bf831b4917a63bff8ccead5037b4a1b: Status 404 returned error can't find the container with id 25472cafa14d86c0a14cf331da39bbdf0bf831b4917a63bff8ccead5037b4a1b
Jul 07 15:09:41.461089 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:41.461050    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b477bdf49-dln4b" event={"ID":"84e8f9e9-dd85-4b04-8594-1521b9a71765","Type":"ContainerStarted","Data":"25472cafa14d86c0a14cf331da39bbdf0bf831b4917a63bff8ccead5037b4a1b"}
Jul 07 15:09:43.474257 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:43.474161    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57b998dcff-nkdvf" event={"ID":"a67bcb42-318f-43a3-b3d5-70dba7ab4915","Type":"ContainerStarted","Data":"81acb5168db1e0272e169a5ba2b0a3e325afa09eb55dbe995f094c802587ee70"}
Jul 07 15:09:43.474807 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:43.474270    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-57b998dcff-nkdvf"
Jul 07 15:09:43.476251 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:43.476223    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b477bdf49-dln4b" event={"ID":"84e8f9e9-dd85-4b04-8594-1521b9a71765","Type":"ContainerStarted","Data":"4b632bdc07b0d0c2a22eb7172d411bdf36215cc02a32fe047fb0dd61c8470706"}
Jul 07 15:09:43.476450 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:43.476282    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5b477bdf49-dln4b"
Jul 07 15:09:43.499422 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:43.499368    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-57b998dcff-nkdvf" podStartSLOduration=1.753008147 podStartE2EDuration="4.499354503s" totalImagesPullingTime="2.746346356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:09:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:09:43.497633989 +0000 UTC m=+551.294170718" watchObservedRunningTime="2026-07-07 15:09:43.499354503 +0000 UTC m=+551.295891231"
Jul 07 15:09:43.515971 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:43.515926    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5b477bdf49-dln4b" podStartSLOduration=0.980167229 podStartE2EDuration="3.515911665s" totalImagesPullingTime="2.535744436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:09:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:09:43.515201948 +0000 UTC m=+551.311738695" watchObservedRunningTime="2026-07-07 15:09:43.515911665 +0000 UTC m=+551.312448393"
Jul 07 15:09:54.487548 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:54.487515    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-57b998dcff-nkdvf"
Jul 07 15:09:54.488063 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:54.487587    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5b477bdf49-dln4b"
Jul 07 15:09:54.560398 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:54.560360    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57b998dcff-nkdvf"]
Jul 07 15:09:54.560653 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:54.560600    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-57b998dcff-nkdvf" podUID="a67bcb42-318f-43a3-b3d5-70dba7ab4915" containerName="manager" containerID="cri-o://81acb5168db1e0272e169a5ba2b0a3e325afa09eb55dbe995f094c802587ee70" gracePeriod=30
Jul 07 15:09:54.817358 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:54.817331    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57b998dcff-nkdvf"
Jul 07 15:09:54.998046 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:54.998008    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a67bcb42-318f-43a3-b3d5-70dba7ab4915-webhook-cert\") pod \"a67bcb42-318f-43a3-b3d5-70dba7ab4915\" (UID: \"a67bcb42-318f-43a3-b3d5-70dba7ab4915\") "
Jul 07 15:09:54.998239 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:54.998090    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9pzs\" (UniqueName: \"kubernetes.io/projected/a67bcb42-318f-43a3-b3d5-70dba7ab4915-kube-api-access-p9pzs\") pod \"a67bcb42-318f-43a3-b3d5-70dba7ab4915\" (UID: \"a67bcb42-318f-43a3-b3d5-70dba7ab4915\") "
Jul 07 15:09:55.000333 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:55.000306    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a67bcb42-318f-43a3-b3d5-70dba7ab4915-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a67bcb42-318f-43a3-b3d5-70dba7ab4915" (UID: "a67bcb42-318f-43a3-b3d5-70dba7ab4915"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:09:55.000428 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:55.000356    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a67bcb42-318f-43a3-b3d5-70dba7ab4915-kube-api-access-p9pzs" (OuterVolumeSpecName: "kube-api-access-p9pzs") pod "a67bcb42-318f-43a3-b3d5-70dba7ab4915" (UID: "a67bcb42-318f-43a3-b3d5-70dba7ab4915"). InnerVolumeSpecName "kube-api-access-p9pzs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:09:55.099526 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:55.099490    2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a67bcb42-318f-43a3-b3d5-70dba7ab4915-webhook-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:09:55.099699 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:55.099543    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9pzs\" (UniqueName: \"kubernetes.io/projected/a67bcb42-318f-43a3-b3d5-70dba7ab4915-kube-api-access-p9pzs\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:09:55.541063 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:55.540966    2567 generic.go:356] "Generic (PLEG): container finished" podID="a67bcb42-318f-43a3-b3d5-70dba7ab4915" containerID="81acb5168db1e0272e169a5ba2b0a3e325afa09eb55dbe995f094c802587ee70" exitCode=0
Jul 07 15:09:55.541063 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:55.541014    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57b998dcff-nkdvf"
Jul 07 15:09:55.541063 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:55.541043    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57b998dcff-nkdvf" event={"ID":"a67bcb42-318f-43a3-b3d5-70dba7ab4915","Type":"ContainerDied","Data":"81acb5168db1e0272e169a5ba2b0a3e325afa09eb55dbe995f094c802587ee70"}
Jul 07 15:09:55.541554 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:55.541103    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57b998dcff-nkdvf" event={"ID":"a67bcb42-318f-43a3-b3d5-70dba7ab4915","Type":"ContainerDied","Data":"7b7da79ba14c355301fb3bff246a6b6f4e468be43359718dc85e090836d7970b"}
Jul 07 15:09:55.541554 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:55.541127    2567 scope.go:117] "RemoveContainer" containerID="81acb5168db1e0272e169a5ba2b0a3e325afa09eb55dbe995f094c802587ee70"
Jul 07 15:09:55.550272 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:55.550251    2567 scope.go:117] "RemoveContainer" containerID="81acb5168db1e0272e169a5ba2b0a3e325afa09eb55dbe995f094c802587ee70"
Jul 07 15:09:55.550542 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:09:55.550520    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81acb5168db1e0272e169a5ba2b0a3e325afa09eb55dbe995f094c802587ee70\": container with ID starting with 81acb5168db1e0272e169a5ba2b0a3e325afa09eb55dbe995f094c802587ee70 not found: ID does not exist" containerID="81acb5168db1e0272e169a5ba2b0a3e325afa09eb55dbe995f094c802587ee70"
Jul 07 15:09:55.550638 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:55.550545    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81acb5168db1e0272e169a5ba2b0a3e325afa09eb55dbe995f094c802587ee70"} err="failed to get container status \"81acb5168db1e0272e169a5ba2b0a3e325afa09eb55dbe995f094c802587ee70\": rpc error: code = NotFound desc = could not find container \"81acb5168db1e0272e169a5ba2b0a3e325afa09eb55dbe995f094c802587ee70\": container with ID starting with 81acb5168db1e0272e169a5ba2b0a3e325afa09eb55dbe995f094c802587ee70 not found: ID does not exist"
Jul 07 15:09:55.565616 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:55.565594    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57b998dcff-nkdvf"]
Jul 07 15:09:55.569280 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:55.569260    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-57b998dcff-nkdvf"]
Jul 07 15:09:56.777498 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:09:56.777462    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a67bcb42-318f-43a3-b3d5-70dba7ab4915" path="/var/lib/kubelet/pods/a67bcb42-318f-43a3-b3d5-70dba7ab4915/volumes"
Jul 07 15:10:16.810029 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:16.809988    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-569b75bc5d-b4jvx"]
Jul 07 15:10:16.810989 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:16.810946    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a67bcb42-318f-43a3-b3d5-70dba7ab4915" containerName="manager"
Jul 07 15:10:16.810989 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:16.810975    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a67bcb42-318f-43a3-b3d5-70dba7ab4915" containerName="manager"
Jul 07 15:10:16.811198 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:16.811188    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a67bcb42-318f-43a3-b3d5-70dba7ab4915" containerName="manager"
Jul 07 15:10:16.840701 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:16.840663    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-569b75bc5d-b4jvx"]
Jul 07 15:10:16.840869 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:16.840799    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-569b75bc5d-b4jvx"
Jul 07 15:10:16.844681 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:16.844640    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\""
Jul 07 15:10:16.844869 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:16.844762    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-l6lq8\""
Jul 07 15:10:16.885115 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:16.885083    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e6a50ef-eec0-4399-bef5-daf059cb5e88-openshift-service-ca\") pod \"maas-api-569b75bc5d-b4jvx\" (UID: \"4e6a50ef-eec0-4399-bef5-daf059cb5e88\") " pod="opendatahub/maas-api-569b75bc5d-b4jvx"
Jul 07 15:10:16.885302 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:16.885124    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e6a50ef-eec0-4399-bef5-daf059cb5e88-maas-api-tls\") pod \"maas-api-569b75bc5d-b4jvx\" (UID: \"4e6a50ef-eec0-4399-bef5-daf059cb5e88\") " pod="opendatahub/maas-api-569b75bc5d-b4jvx"
Jul 07 15:10:16.885302 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:16.885166    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sng2t\" (UniqueName: \"kubernetes.io/projected/4e6a50ef-eec0-4399-bef5-daf059cb5e88-kube-api-access-sng2t\") pod \"maas-api-569b75bc5d-b4jvx\" (UID: \"4e6a50ef-eec0-4399-bef5-daf059cb5e88\") " pod="opendatahub/maas-api-569b75bc5d-b4jvx"
Jul 07 15:10:16.986089 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:16.986048    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sng2t\" (UniqueName: \"kubernetes.io/projected/4e6a50ef-eec0-4399-bef5-daf059cb5e88-kube-api-access-sng2t\") pod \"maas-api-569b75bc5d-b4jvx\" (UID: \"4e6a50ef-eec0-4399-bef5-daf059cb5e88\") " pod="opendatahub/maas-api-569b75bc5d-b4jvx"
Jul 07 15:10:16.986289 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:16.986143    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e6a50ef-eec0-4399-bef5-daf059cb5e88-openshift-service-ca\") pod \"maas-api-569b75bc5d-b4jvx\" (UID: \"4e6a50ef-eec0-4399-bef5-daf059cb5e88\") " pod="opendatahub/maas-api-569b75bc5d-b4jvx"
Jul 07 15:10:16.986289 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:16.986176    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e6a50ef-eec0-4399-bef5-daf059cb5e88-maas-api-tls\") pod \"maas-api-569b75bc5d-b4jvx\" (UID: \"4e6a50ef-eec0-4399-bef5-daf059cb5e88\") " pod="opendatahub/maas-api-569b75bc5d-b4jvx"
Jul 07 15:10:16.986805 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:16.986783    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e6a50ef-eec0-4399-bef5-daf059cb5e88-openshift-service-ca\") pod \"maas-api-569b75bc5d-b4jvx\" (UID: \"4e6a50ef-eec0-4399-bef5-daf059cb5e88\") " pod="opendatahub/maas-api-569b75bc5d-b4jvx"
Jul 07 15:10:16.988639 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:16.988617    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e6a50ef-eec0-4399-bef5-daf059cb5e88-maas-api-tls\") pod \"maas-api-569b75bc5d-b4jvx\" (UID: \"4e6a50ef-eec0-4399-bef5-daf059cb5e88\") " pod="opendatahub/maas-api-569b75bc5d-b4jvx"
Jul 07 15:10:16.997680 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:16.997654    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sng2t\" (UniqueName: \"kubernetes.io/projected/4e6a50ef-eec0-4399-bef5-daf059cb5e88-kube-api-access-sng2t\") pod \"maas-api-569b75bc5d-b4jvx\" (UID: \"4e6a50ef-eec0-4399-bef5-daf059cb5e88\") " pod="opendatahub/maas-api-569b75bc5d-b4jvx"
Jul 07 15:10:17.152648 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:17.152566    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-569b75bc5d-b4jvx"
Jul 07 15:10:17.313731 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:17.313701    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-569b75bc5d-b4jvx"]
Jul 07 15:10:17.317239 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:10:17.317211    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e6a50ef_eec0_4399_bef5_daf059cb5e88.slice/crio-06c7393404ac44dd6a6ca6ccb35ecf5fc1d4d6f4ff87087e0bac633efed8f164 WatchSource:0}: Error finding container 06c7393404ac44dd6a6ca6ccb35ecf5fc1d4d6f4ff87087e0bac633efed8f164: Status 404 returned error can't find the container with id 06c7393404ac44dd6a6ca6ccb35ecf5fc1d4d6f4ff87087e0bac633efed8f164
Jul 07 15:10:17.512141 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:17.512061    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-6zf6z"]
Jul 07 15:10:17.521426 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:17.521401    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-6zf6z"
Jul 07 15:10:17.528152 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:17.528123    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-xsgrr\""
Jul 07 15:10:17.528700 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:17.528646    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 07 15:10:17.530444 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:17.530420    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-6zf6z"]
Jul 07 15:10:17.568896 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:17.568860    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-gt4qs"]
Jul 07 15:10:17.572533 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:17.572515    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-gt4qs"
Jul 07 15:10:17.584263 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:17.584234    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-gt4qs"]
Jul 07 15:10:17.591732 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:17.591705    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8ce75f8-4730-4bac-9374-04d5076960e8-plugins-config-volume\") pod \"payload-pre-processing-7964457696-6zf6z\" (UID: \"d8ce75f8-4730-4bac-9374-04d5076960e8\") " pod="openshift-ingress/payload-pre-processing-7964457696-6zf6z"
Jul 07 15:10:17.591889 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:17.591764    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzjr4\" (UniqueName: \"kubernetes.io/projected/d8ce75f8-4730-4bac-9374-04d5076960e8-kube-api-access-pzjr4\") pod \"payload-pre-processing-7964457696-6zf6z\" (UID: \"d8ce75f8-4730-4bac-9374-04d5076960e8\") " pod="openshift-ingress/payload-pre-processing-7964457696-6zf6z"
Jul 07 15:10:17.649247 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:17.649211    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-569b75bc5d-b4jvx" event={"ID":"4e6a50ef-eec0-4399-bef5-daf059cb5e88","Type":"ContainerStarted","Data":"06c7393404ac44dd6a6ca6ccb35ecf5fc1d4d6f4ff87087e0bac633efed8f164"}
Jul 07 15:10:17.692798 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:17.692763    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8ce75f8-4730-4bac-9374-04d5076960e8-plugins-config-volume\") pod \"payload-pre-processing-7964457696-6zf6z\" (UID: \"d8ce75f8-4730-4bac-9374-04d5076960e8\") " pod="openshift-ingress/payload-pre-processing-7964457696-6zf6z"
Jul 07 15:10:17.692798 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:17.692802    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9s942\" (UniqueName: \"kubernetes.io/projected/8ee89224-43b1-45b8-bb94-6f1fe3d4f949-kube-api-access-9s942\") pod \"payload-processing-58f86c7798-gt4qs\" (UID: \"8ee89224-43b1-45b8-bb94-6f1fe3d4f949\") " pod="openshift-ingress/payload-processing-58f86c7798-gt4qs"
Jul 07 15:10:17.693055 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:17.692874    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzjr4\" (UniqueName: \"kubernetes.io/projected/d8ce75f8-4730-4bac-9374-04d5076960e8-kube-api-access-pzjr4\") pod \"payload-pre-processing-7964457696-6zf6z\" (UID: \"d8ce75f8-4730-4bac-9374-04d5076960e8\") " pod="openshift-ingress/payload-pre-processing-7964457696-6zf6z"
Jul 07 15:10:17.693055 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:17.692890    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ee89224-43b1-45b8-bb94-6f1fe3d4f949-plugins-config-volume\") pod \"payload-processing-58f86c7798-gt4qs\" (UID: \"8ee89224-43b1-45b8-bb94-6f1fe3d4f949\") " pod="openshift-ingress/payload-processing-58f86c7798-gt4qs"
Jul 07 15:10:17.693508 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:17.693486    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8ce75f8-4730-4bac-9374-04d5076960e8-plugins-config-volume\") pod \"payload-pre-processing-7964457696-6zf6z\" (UID: \"d8ce75f8-4730-4bac-9374-04d5076960e8\") " pod="openshift-ingress/payload-pre-processing-7964457696-6zf6z"
Jul 07 15:10:17.705136 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:17.705107    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzjr4\" (UniqueName: \"kubernetes.io/projected/d8ce75f8-4730-4bac-9374-04d5076960e8-kube-api-access-pzjr4\") pod \"payload-pre-processing-7964457696-6zf6z\" (UID: \"d8ce75f8-4730-4bac-9374-04d5076960e8\") " pod="openshift-ingress/payload-pre-processing-7964457696-6zf6z"
Jul 07 15:10:17.794084 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:17.793989    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9s942\" (UniqueName: \"kubernetes.io/projected/8ee89224-43b1-45b8-bb94-6f1fe3d4f949-kube-api-access-9s942\") pod \"payload-processing-58f86c7798-gt4qs\" (UID: \"8ee89224-43b1-45b8-bb94-6f1fe3d4f949\") " pod="openshift-ingress/payload-processing-58f86c7798-gt4qs"
Jul 07 15:10:17.794084 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:17.794050    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ee89224-43b1-45b8-bb94-6f1fe3d4f949-plugins-config-volume\") pod \"payload-processing-58f86c7798-gt4qs\" (UID: \"8ee89224-43b1-45b8-bb94-6f1fe3d4f949\") " pod="openshift-ingress/payload-processing-58f86c7798-gt4qs"
Jul 07 15:10:17.794736 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:17.794715    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ee89224-43b1-45b8-bb94-6f1fe3d4f949-plugins-config-volume\") pod \"payload-processing-58f86c7798-gt4qs\" (UID: \"8ee89224-43b1-45b8-bb94-6f1fe3d4f949\") " pod="openshift-ingress/payload-processing-58f86c7798-gt4qs"
Jul 07 15:10:17.804517 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:17.804491    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9s942\" (UniqueName: \"kubernetes.io/projected/8ee89224-43b1-45b8-bb94-6f1fe3d4f949-kube-api-access-9s942\") pod \"payload-processing-58f86c7798-gt4qs\" (UID: \"8ee89224-43b1-45b8-bb94-6f1fe3d4f949\") " pod="openshift-ingress/payload-processing-58f86c7798-gt4qs"
Jul 07 15:10:17.833634 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:17.833598    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-6zf6z"
Jul 07 15:10:17.883204 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:17.883169    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-gt4qs"
Jul 07 15:10:18.046500 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:18.046414    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-6zf6z"]
Jul 07 15:10:18.057434 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:10:18.057394    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8ce75f8_4730_4bac_9374_04d5076960e8.slice/crio-575234eeec04228dd81422fa71b6fde131838db82447a33fe44a379d0257bb6d WatchSource:0}: Error finding container 575234eeec04228dd81422fa71b6fde131838db82447a33fe44a379d0257bb6d: Status 404 returned error can't find the container with id 575234eeec04228dd81422fa71b6fde131838db82447a33fe44a379d0257bb6d
Jul 07 15:10:18.115389 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:18.115311    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-gt4qs"]
Jul 07 15:10:18.139262 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:10:18.139142    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ee89224_43b1_45b8_bb94_6f1fe3d4f949.slice/crio-51e6b49050458856dc7f5b96f784a63e1efee41655098fbbe82c2ba2051c68a6 WatchSource:0}: Error finding container 51e6b49050458856dc7f5b96f784a63e1efee41655098fbbe82c2ba2051c68a6: Status 404 returned error can't find the container with id 51e6b49050458856dc7f5b96f784a63e1efee41655098fbbe82c2ba2051c68a6
Jul 07 15:10:18.655254 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:18.655211    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-gt4qs" event={"ID":"8ee89224-43b1-45b8-bb94-6f1fe3d4f949","Type":"ContainerStarted","Data":"51e6b49050458856dc7f5b96f784a63e1efee41655098fbbe82c2ba2051c68a6"}
Jul 07 15:10:18.656395 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:18.656372    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-6zf6z" event={"ID":"d8ce75f8-4730-4bac-9374-04d5076960e8","Type":"ContainerStarted","Data":"575234eeec04228dd81422fa71b6fde131838db82447a33fe44a379d0257bb6d"}
Jul 07 15:10:20.669056 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:20.669016    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-6zf6z" event={"ID":"d8ce75f8-4730-4bac-9374-04d5076960e8","Type":"ContainerStarted","Data":"f2037916c50070aab93fd486198a298442196fef23a26731da22239cb063f699"}
Jul 07 15:10:20.669534 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:20.669246    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-6zf6z"
Jul 07 15:10:20.670728 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:20.670705    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-569b75bc5d-b4jvx" event={"ID":"4e6a50ef-eec0-4399-bef5-daf059cb5e88","Type":"ContainerStarted","Data":"6cb08fab80072887b7dc99864fe090bef44bfc44ecf664c73679cde94b3d4930"}
Jul 07 15:10:20.670815 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:20.670797    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-569b75bc5d-b4jvx"
Jul 07 15:10:20.672337 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:20.672311    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-gt4qs" event={"ID":"8ee89224-43b1-45b8-bb94-6f1fe3d4f949","Type":"ContainerStarted","Data":"a3b30a636235bde0fef86800a9ace9a7c59c5e3be3aeaea4b5b7e7d15461c13f"}
Jul 07 15:10:20.672447 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:20.672433    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-gt4qs"
Jul 07 15:10:20.689047 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:20.688997    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-6zf6z" podStartSLOduration=1.762984297 podStartE2EDuration="3.688981128s" totalImagesPullingTime="1.925996831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:10:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:10:20.687002407 +0000 UTC m=+588.483539153" watchObservedRunningTime="2026-07-07 15:10:20.688981128 +0000 UTC m=+588.485517856"
Jul 07 15:10:20.710266 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:20.710210    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-569b75bc5d-b4jvx" podStartSLOduration=2.120518815 podStartE2EDuration="4.710189894s" totalImagesPullingTime="2.589671079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:10:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:10:20.709182915 +0000 UTC m=+588.505719664" watchObservedRunningTime="2026-07-07 15:10:20.710189894 +0000 UTC m=+588.506726675"
Jul 07 15:10:20.728226 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:20.728155    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-gt4qs" podStartSLOduration=1.886364344 podStartE2EDuration="3.728131549s" totalImagesPullingTime="1.841767205s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:10:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:10:20.726584116 +0000 UTC m=+588.523120844" watchObservedRunningTime="2026-07-07 15:10:20.728131549 +0000 UTC m=+588.524668279"
Jul 07 15:10:25.704053 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:25.704020    2567 generic.go:356] "Generic (PLEG): container finished" podID="497c7b71-8fa8-44b0-b75c-60f11cc9bd6c" containerID="ff431a11a520461fa09b63cea27d440558281b8f53e7ad4aab85c60bc93fe160" exitCode=0
Jul 07 15:10:25.704627 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:25.704100    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj" event={"ID":"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c","Type":"ContainerDied","Data":"ff431a11a520461fa09b63cea27d440558281b8f53e7ad4aab85c60bc93fe160"}
Jul 07 15:10:25.704700 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:25.704627    2567 scope.go:117] "RemoveContainer" containerID="ff431a11a520461fa09b63cea27d440558281b8f53e7ad4aab85c60bc93fe160"
Jul 07 15:10:25.705643 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:25.705619    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:10:25.705696 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:25.705688    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:10:25.705734 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:25.705716    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:10:26.683119 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:26.683091    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-569b75bc5d-b4jvx"
Jul 07 15:10:26.711535 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:26.711506    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj" event={"ID":"497c7b71-8fa8-44b0-b75c-60f11cc9bd6c","Type":"ContainerStarted","Data":"7e8c51deb041ef026a45fc5dd7918962088a46a71e5a47ecfd522d27c5300466"}
Jul 07 15:10:26.931281 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:26.931241    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:10:26.931725 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:26.931693    2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-q9trj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Jul 07 15:10:26.931868 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:26.931772    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj" podUID="497c7b71-8fa8-44b0-b75c-60f11cc9bd6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Jul 07 15:10:27.932212 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:27.932163    2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-q9trj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Jul 07 15:10:27.932722 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:27.932249    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj" podUID="497c7b71-8fa8-44b0-b75c-60f11cc9bd6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Jul 07 15:10:28.931801 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:28.931768    2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-q9trj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Jul 07 15:10:28.932113 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:28.932082    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj" podUID="497c7b71-8fa8-44b0-b75c-60f11cc9bd6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Jul 07 15:10:29.932173 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:29.932133    2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-q9trj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Jul 07 15:10:29.932684 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:29.932207    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj" podUID="497c7b71-8fa8-44b0-b75c-60f11cc9bd6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Jul 07 15:10:30.935665 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:30.935629    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:10:30.936158 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:30.936023    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:10:30.936564 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:30.936544    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-q9trj"
Jul 07 15:10:31.679391 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:31.679354    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-gt4qs"
Jul 07 15:10:31.679568 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:31.679440    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-6zf6z"
Jul 07 15:10:32.679870 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:32.679819    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5r9rp_1c51cd93-b0ea-4aa1-8678-a4c833b2d37e/console-operator/1.log"
Jul 07 15:10:32.680564 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:32.680547    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5r9rp_1c51cd93-b0ea-4aa1-8678-a4c833b2d37e/console-operator/1.log"
Jul 07 15:10:52.274675 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.274635    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx"]
Jul 07 15:10:52.280345 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.280319    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx"
Jul 07 15:10:52.283313 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.283286    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 07 15:10:52.284711 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.284690    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 07 15:10:52.284844 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.284714    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 07 15:10:52.284844 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.284695    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-tjdd8\""
Jul 07 15:10:52.289992 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.289970    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx"]
Jul 07 15:10:52.324038 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.324004    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e69ed3f1-1661-494b-829a-5b91cf866103-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5h6hx\" (UID: \"e69ed3f1-1661-494b-829a-5b91cf866103\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx"
Jul 07 15:10:52.324215 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.324045    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e69ed3f1-1661-494b-829a-5b91cf866103-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5h6hx\" (UID: \"e69ed3f1-1661-494b-829a-5b91cf866103\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx"
Jul 07 15:10:52.324215 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.324072    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e69ed3f1-1661-494b-829a-5b91cf866103-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5h6hx\" (UID: \"e69ed3f1-1661-494b-829a-5b91cf866103\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx"
Jul 07 15:10:52.324215 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.324110    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pjdn\" (UniqueName: \"kubernetes.io/projected/e69ed3f1-1661-494b-829a-5b91cf866103-kube-api-access-9pjdn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5h6hx\" (UID: \"e69ed3f1-1661-494b-829a-5b91cf866103\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx"
Jul 07 15:10:52.324215 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.324211    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e69ed3f1-1661-494b-829a-5b91cf866103-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5h6hx\" (UID: \"e69ed3f1-1661-494b-829a-5b91cf866103\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx"
Jul 07 15:10:52.324410 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.324270    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e69ed3f1-1661-494b-829a-5b91cf866103-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5h6hx\" (UID: \"e69ed3f1-1661-494b-829a-5b91cf866103\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx"
Jul 07 15:10:52.425120 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.425090    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e69ed3f1-1661-494b-829a-5b91cf866103-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5h6hx\" (UID: \"e69ed3f1-1661-494b-829a-5b91cf866103\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx"
Jul 07 15:10:52.425293 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.425143    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e69ed3f1-1661-494b-829a-5b91cf866103-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5h6hx\" (UID: \"e69ed3f1-1661-494b-829a-5b91cf866103\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx"
Jul 07 15:10:52.425293 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.425176    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e69ed3f1-1661-494b-829a-5b91cf866103-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5h6hx\" (UID: \"e69ed3f1-1661-494b-829a-5b91cf866103\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx"
Jul 07 15:10:52.425293 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.425203    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e69ed3f1-1661-494b-829a-5b91cf866103-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5h6hx\" (UID: \"e69ed3f1-1661-494b-829a-5b91cf866103\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx"
Jul 07 15:10:52.425293 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.425224    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e69ed3f1-1661-494b-829a-5b91cf866103-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5h6hx\" (UID: \"e69ed3f1-1661-494b-829a-5b91cf866103\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx"
Jul 07 15:10:52.425293 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.425250    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pjdn\" (UniqueName: \"kubernetes.io/projected/e69ed3f1-1661-494b-829a-5b91cf866103-kube-api-access-9pjdn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5h6hx\" (UID: \"e69ed3f1-1661-494b-829a-5b91cf866103\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx"
Jul 07 15:10:52.425563 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.425546    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e69ed3f1-1661-494b-829a-5b91cf866103-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5h6hx\" (UID: \"e69ed3f1-1661-494b-829a-5b91cf866103\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx"
Jul 07 15:10:52.425635 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.425553    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e69ed3f1-1661-494b-829a-5b91cf866103-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5h6hx\" (UID: \"e69ed3f1-1661-494b-829a-5b91cf866103\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx"
Jul 07 15:10:52.425635 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.425616    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e69ed3f1-1661-494b-829a-5b91cf866103-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5h6hx\" (UID: \"e69ed3f1-1661-494b-829a-5b91cf866103\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx"
Jul 07 15:10:52.427682 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.427657    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e69ed3f1-1661-494b-829a-5b91cf866103-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5h6hx\" (UID: \"e69ed3f1-1661-494b-829a-5b91cf866103\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx"
Jul 07 15:10:52.427926 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.427906    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e69ed3f1-1661-494b-829a-5b91cf866103-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5h6hx\" (UID: \"e69ed3f1-1661-494b-829a-5b91cf866103\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx"
Jul 07 15:10:52.435164 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.435143    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pjdn\" (UniqueName: \"kubernetes.io/projected/e69ed3f1-1661-494b-829a-5b91cf866103-kube-api-access-9pjdn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5h6hx\" (UID: \"e69ed3f1-1661-494b-829a-5b91cf866103\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx"
Jul 07 15:10:52.594841 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.594807    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx"
Jul 07 15:10:52.739972 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.739933    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx"]
Jul 07 15:10:52.742462 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:10:52.742434    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode69ed3f1_1661_494b_829a_5b91cf866103.slice/crio-d0704fed23097662a528533a5d34cafd1b03b37bea2fdf406e81cc02eefe2ae7 WatchSource:0}: Error finding container d0704fed23097662a528533a5d34cafd1b03b37bea2fdf406e81cc02eefe2ae7: Status 404 returned error can't find the container with id d0704fed23097662a528533a5d34cafd1b03b37bea2fdf406e81cc02eefe2ae7
Jul 07 15:10:52.743963 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.743943    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 15:10:52.848486 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:52.848406    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx" event={"ID":"e69ed3f1-1661-494b-829a-5b91cf866103","Type":"ContainerStarted","Data":"d0704fed23097662a528533a5d34cafd1b03b37bea2fdf406e81cc02eefe2ae7"}
Jul 07 15:10:54.864299 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:54.864255    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx" event={"ID":"e69ed3f1-1661-494b-829a-5b91cf866103","Type":"ContainerStarted","Data":"0fde557108140d94246ef138337567e6ad27ffd64ce94863736a1ea7e1661399"}
Jul 07 15:10:54.886337 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:10:54.886279    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx" podStartSLOduration=1.679652854 podStartE2EDuration="2.886262556s" totalImagesPullingTime="1.206609702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:10:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:10:54.885322834 +0000 UTC m=+622.681859563" watchObservedRunningTime="2026-07-07 15:10:54.886262556 +0000 UTC m=+622.682799284"
Jul 07 15:11:02.595429 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:02.595384    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx"
Jul 07 15:11:02.595997 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:02.595451    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx"
Jul 07 15:11:02.608568 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:02.608540    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx"
Jul 07 15:11:02.915036 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:02.914953    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5h6hx"
Jul 07 15:11:17.870151 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:17.870121    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85"]
Jul 07 15:11:17.874129 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:17.874106    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85"
Jul 07 15:11:17.877204 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:17.877180    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 07 15:11:17.884231 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:17.884205    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85"]
Jul 07 15:11:17.966819 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:17.966782    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39c6c995-1620-46c6-bfe2-6c2d0e402ec0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85\" (UID: \"39c6c995-1620-46c6-bfe2-6c2d0e402ec0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85"
Jul 07 15:11:17.966819 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:17.966824    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzl2k\" (UniqueName: \"kubernetes.io/projected/39c6c995-1620-46c6-bfe2-6c2d0e402ec0-kube-api-access-rzl2k\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85\" (UID: \"39c6c995-1620-46c6-bfe2-6c2d0e402ec0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85"
Jul 07 15:11:17.967032 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:17.966890    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39c6c995-1620-46c6-bfe2-6c2d0e402ec0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85\" (UID: \"39c6c995-1620-46c6-bfe2-6c2d0e402ec0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85"
Jul 07 15:11:17.967032 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:17.966919    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39c6c995-1620-46c6-bfe2-6c2d0e402ec0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85\" (UID: \"39c6c995-1620-46c6-bfe2-6c2d0e402ec0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85"
Jul 07 15:11:17.967032 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:17.966936    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39c6c995-1620-46c6-bfe2-6c2d0e402ec0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85\" (UID: \"39c6c995-1620-46c6-bfe2-6c2d0e402ec0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85"
Jul 07 15:11:17.967032 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:17.966953    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39c6c995-1620-46c6-bfe2-6c2d0e402ec0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85\" (UID: \"39c6c995-1620-46c6-bfe2-6c2d0e402ec0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85"
Jul 07 15:11:18.068127 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:18.068088    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39c6c995-1620-46c6-bfe2-6c2d0e402ec0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85\" (UID: \"39c6c995-1620-46c6-bfe2-6c2d0e402ec0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85"
Jul 07 15:11:18.068127 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:18.068133    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzl2k\" (UniqueName: \"kubernetes.io/projected/39c6c995-1620-46c6-bfe2-6c2d0e402ec0-kube-api-access-rzl2k\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85\" (UID: \"39c6c995-1620-46c6-bfe2-6c2d0e402ec0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85"
Jul 07 15:11:18.068392 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:18.068171    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39c6c995-1620-46c6-bfe2-6c2d0e402ec0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85\" (UID: \"39c6c995-1620-46c6-bfe2-6c2d0e402ec0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85"
Jul 07 15:11:18.068392 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:18.068200    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39c6c995-1620-46c6-bfe2-6c2d0e402ec0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85\" (UID: \"39c6c995-1620-46c6-bfe2-6c2d0e402ec0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85"
Jul 07 15:11:18.068392 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:18.068320    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39c6c995-1620-46c6-bfe2-6c2d0e402ec0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85\" (UID: \"39c6c995-1620-46c6-bfe2-6c2d0e402ec0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85"
Jul 07 15:11:18.068392 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:18.068349    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39c6c995-1620-46c6-bfe2-6c2d0e402ec0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85\" (UID: \"39c6c995-1620-46c6-bfe2-6c2d0e402ec0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85"
Jul 07 15:11:18.068596 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:18.068563    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39c6c995-1620-46c6-bfe2-6c2d0e402ec0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85\" (UID: \"39c6c995-1620-46c6-bfe2-6c2d0e402ec0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85"
Jul 07 15:11:18.068746 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:18.068728    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39c6c995-1620-46c6-bfe2-6c2d0e402ec0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85\" (UID: \"39c6c995-1620-46c6-bfe2-6c2d0e402ec0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85"
Jul 07 15:11:18.068808 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:18.068773    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39c6c995-1620-46c6-bfe2-6c2d0e402ec0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85\" (UID: \"39c6c995-1620-46c6-bfe2-6c2d0e402ec0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85"
Jul 07 15:11:18.070691 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:18.070656    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39c6c995-1620-46c6-bfe2-6c2d0e402ec0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85\" (UID: \"39c6c995-1620-46c6-bfe2-6c2d0e402ec0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85"
Jul 07 15:11:18.070966 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:18.070938    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39c6c995-1620-46c6-bfe2-6c2d0e402ec0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85\" (UID: \"39c6c995-1620-46c6-bfe2-6c2d0e402ec0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85"
Jul 07 15:11:18.078300 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:18.078273    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzl2k\" (UniqueName: \"kubernetes.io/projected/39c6c995-1620-46c6-bfe2-6c2d0e402ec0-kube-api-access-rzl2k\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85\" (UID: \"39c6c995-1620-46c6-bfe2-6c2d0e402ec0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85"
Jul 07 15:11:18.187962 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:18.187864    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85"
Jul 07 15:11:18.333650 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:18.333620    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85"]
Jul 07 15:11:18.335580 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:11:18.335553    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39c6c995_1620_46c6_bfe2_6c2d0e402ec0.slice/crio-17fc96c83ed0a55e3b863586fec311114d832c18a6a2dc5e27f839c548b3fdf9 WatchSource:0}: Error finding container 17fc96c83ed0a55e3b863586fec311114d832c18a6a2dc5e27f839c548b3fdf9: Status 404 returned error can't find the container with id 17fc96c83ed0a55e3b863586fec311114d832c18a6a2dc5e27f839c548b3fdf9
Jul 07 15:11:18.986667 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:18.986582    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85" event={"ID":"39c6c995-1620-46c6-bfe2-6c2d0e402ec0","Type":"ContainerStarted","Data":"e65dfb882f5217e017574743e35d7345f8a5c06c4e4e3e8c6778f68d90a2c4de"}
Jul 07 15:11:18.986667 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:18.986636    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85" event={"ID":"39c6c995-1620-46c6-bfe2-6c2d0e402ec0","Type":"ContainerStarted","Data":"17fc96c83ed0a55e3b863586fec311114d832c18a6a2dc5e27f839c548b3fdf9"}
Jul 07 15:11:19.013470 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:19.013412    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85" podStartSLOduration=1.669354209 podStartE2EDuration="2.013394809s" totalImagesPullingTime="344.0406ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:11:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:11:19.013064229 +0000 UTC m=+646.809600981" watchObservedRunningTime="2026-07-07 15:11:19.013394809 +0000 UTC m=+646.809931536"
Jul 07 15:11:28.188165 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:28.188129    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85"
Jul 07 15:11:28.188573 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:28.188201    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85"
Jul 07 15:11:28.200677 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:28.200651    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85"
Jul 07 15:11:29.047721 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:11:29.047694    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85"
Jul 07 15:13:18.227560 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:13:18.227521    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5b586fb75d-2j8cw"]
Jul 07 15:13:18.228493 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:13:18.228439    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5b586fb75d-2j8cw" podUID="6b35acac-02b1-4c22-a6ee-6bc321d7d0b0" containerName="authorino" containerID="cri-o://1da112a5199df80e33f14009a4655a2b3ff93e919ad311cfdcba1c7e10ddc559" gracePeriod=30
Jul 07 15:13:18.485472 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:13:18.485400    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b586fb75d-2j8cw"
Jul 07 15:13:18.583716 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:13:18.583667    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6b35acac-02b1-4c22-a6ee-6bc321d7d0b0-tls-cert\") pod \"6b35acac-02b1-4c22-a6ee-6bc321d7d0b0\" (UID: \"6b35acac-02b1-4c22-a6ee-6bc321d7d0b0\") "
Jul 07 15:13:18.583716 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:13:18.583729    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkjrs\" (UniqueName: \"kubernetes.io/projected/6b35acac-02b1-4c22-a6ee-6bc321d7d0b0-kube-api-access-pkjrs\") pod \"6b35acac-02b1-4c22-a6ee-6bc321d7d0b0\" (UID: \"6b35acac-02b1-4c22-a6ee-6bc321d7d0b0\") "
Jul 07 15:13:18.585935 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:13:18.585901    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b35acac-02b1-4c22-a6ee-6bc321d7d0b0-kube-api-access-pkjrs" (OuterVolumeSpecName: "kube-api-access-pkjrs") pod "6b35acac-02b1-4c22-a6ee-6bc321d7d0b0" (UID: "6b35acac-02b1-4c22-a6ee-6bc321d7d0b0"). InnerVolumeSpecName "kube-api-access-pkjrs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:13:18.595378 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:13:18.595342    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b35acac-02b1-4c22-a6ee-6bc321d7d0b0-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "6b35acac-02b1-4c22-a6ee-6bc321d7d0b0" (UID: "6b35acac-02b1-4c22-a6ee-6bc321d7d0b0"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:13:18.608233 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:13:18.608202    2567 generic.go:356] "Generic (PLEG): container finished" podID="6b35acac-02b1-4c22-a6ee-6bc321d7d0b0" containerID="1da112a5199df80e33f14009a4655a2b3ff93e919ad311cfdcba1c7e10ddc559" exitCode=0
Jul 07 15:13:18.608391 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:13:18.608242    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b586fb75d-2j8cw"
Jul 07 15:13:18.608391 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:13:18.608329    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5b586fb75d-2j8cw" event={"ID":"6b35acac-02b1-4c22-a6ee-6bc321d7d0b0","Type":"ContainerDied","Data":"1da112a5199df80e33f14009a4655a2b3ff93e919ad311cfdcba1c7e10ddc559"}
Jul 07 15:13:18.608391 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:13:18.608374    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5b586fb75d-2j8cw" event={"ID":"6b35acac-02b1-4c22-a6ee-6bc321d7d0b0","Type":"ContainerDied","Data":"898d9f71f31383333a440eab05b23b71a0e06e778d95746d08439c073665ff4b"}
Jul 07 15:13:18.608554 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:13:18.608395    2567 scope.go:117] "RemoveContainer" containerID="1da112a5199df80e33f14009a4655a2b3ff93e919ad311cfdcba1c7e10ddc559"
Jul 07 15:13:18.619123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:13:18.619102    2567 scope.go:117] "RemoveContainer" containerID="1da112a5199df80e33f14009a4655a2b3ff93e919ad311cfdcba1c7e10ddc559"
Jul 07 15:13:18.619452 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:13:18.619429    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1da112a5199df80e33f14009a4655a2b3ff93e919ad311cfdcba1c7e10ddc559\": container with ID starting with 1da112a5199df80e33f14009a4655a2b3ff93e919ad311cfdcba1c7e10ddc559 not found: ID does not exist" containerID="1da112a5199df80e33f14009a4655a2b3ff93e919ad311cfdcba1c7e10ddc559"
Jul 07 15:13:18.619535 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:13:18.619463    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1da112a5199df80e33f14009a4655a2b3ff93e919ad311cfdcba1c7e10ddc559"} err="failed to get container status \"1da112a5199df80e33f14009a4655a2b3ff93e919ad311cfdcba1c7e10ddc559\": rpc error: code = NotFound desc = could not find container \"1da112a5199df80e33f14009a4655a2b3ff93e919ad311cfdcba1c7e10ddc559\": container with ID starting with 1da112a5199df80e33f14009a4655a2b3ff93e919ad311cfdcba1c7e10ddc559 not found: ID does not exist"
Jul 07 15:13:18.631570 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:13:18.631537    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5b586fb75d-2j8cw"]
Jul 07 15:13:18.636007 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:13:18.635980    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5b586fb75d-2j8cw"]
Jul 07 15:13:18.684756 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:13:18.684724    2567 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6b35acac-02b1-4c22-a6ee-6bc321d7d0b0-tls-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:13:18.684756 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:13:18.684750    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkjrs\" (UniqueName: \"kubernetes.io/projected/6b35acac-02b1-4c22-a6ee-6bc321d7d0b0-kube-api-access-pkjrs\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:13:18.778529 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:13:18.778450    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b35acac-02b1-4c22-a6ee-6bc321d7d0b0" path="/var/lib/kubelet/pods/6b35acac-02b1-4c22-a6ee-6bc321d7d0b0/volumes"
Jul 07 15:14:28.970736 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:28.970702    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b477bdf49-dln4b"]
Jul 07 15:14:28.971377 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:28.971002    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-5b477bdf49-dln4b" podUID="84e8f9e9-dd85-4b04-8594-1521b9a71765" containerName="manager" containerID="cri-o://4b632bdc07b0d0c2a22eb7172d411bdf36215cc02a32fe047fb0dd61c8470706" gracePeriod=30
Jul 07 15:14:29.231140 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:29.231079    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b477bdf49-dln4b"
Jul 07 15:14:29.330948 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:29.330910    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84e8f9e9-dd85-4b04-8594-1521b9a71765-webhook-cert\") pod \"84e8f9e9-dd85-4b04-8594-1521b9a71765\" (UID: \"84e8f9e9-dd85-4b04-8594-1521b9a71765\") "
Jul 07 15:14:29.331148 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:29.331030    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dlvr\" (UniqueName: \"kubernetes.io/projected/84e8f9e9-dd85-4b04-8594-1521b9a71765-kube-api-access-5dlvr\") pod \"84e8f9e9-dd85-4b04-8594-1521b9a71765\" (UID: \"84e8f9e9-dd85-4b04-8594-1521b9a71765\") "
Jul 07 15:14:29.333184 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:29.333150    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84e8f9e9-dd85-4b04-8594-1521b9a71765-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "84e8f9e9-dd85-4b04-8594-1521b9a71765" (UID: "84e8f9e9-dd85-4b04-8594-1521b9a71765"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:14:29.333184 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:29.333174    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84e8f9e9-dd85-4b04-8594-1521b9a71765-kube-api-access-5dlvr" (OuterVolumeSpecName: "kube-api-access-5dlvr") pod "84e8f9e9-dd85-4b04-8594-1521b9a71765" (UID: "84e8f9e9-dd85-4b04-8594-1521b9a71765"). InnerVolumeSpecName "kube-api-access-5dlvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:14:29.431876 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:29.431814    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dlvr\" (UniqueName: \"kubernetes.io/projected/84e8f9e9-dd85-4b04-8594-1521b9a71765-kube-api-access-5dlvr\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:14:29.431876 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:29.431870    2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84e8f9e9-dd85-4b04-8594-1521b9a71765-webhook-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:14:29.972820 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:29.972778    2567 generic.go:356] "Generic (PLEG): container finished" podID="84e8f9e9-dd85-4b04-8594-1521b9a71765" containerID="4b632bdc07b0d0c2a22eb7172d411bdf36215cc02a32fe047fb0dd61c8470706" exitCode=0
Jul 07 15:14:29.973316 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:29.972853    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b477bdf49-dln4b"
Jul 07 15:14:29.973316 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:29.972866    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b477bdf49-dln4b" event={"ID":"84e8f9e9-dd85-4b04-8594-1521b9a71765","Type":"ContainerDied","Data":"4b632bdc07b0d0c2a22eb7172d411bdf36215cc02a32fe047fb0dd61c8470706"}
Jul 07 15:14:29.973316 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:29.972961    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b477bdf49-dln4b" event={"ID":"84e8f9e9-dd85-4b04-8594-1521b9a71765","Type":"ContainerDied","Data":"25472cafa14d86c0a14cf331da39bbdf0bf831b4917a63bff8ccead5037b4a1b"}
Jul 07 15:14:29.973316 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:29.972997    2567 scope.go:117] "RemoveContainer" containerID="4b632bdc07b0d0c2a22eb7172d411bdf36215cc02a32fe047fb0dd61c8470706"
Jul 07 15:14:29.982878 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:29.982862    2567 scope.go:117] "RemoveContainer" containerID="4b632bdc07b0d0c2a22eb7172d411bdf36215cc02a32fe047fb0dd61c8470706"
Jul 07 15:14:29.983137 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:14:29.983120    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b632bdc07b0d0c2a22eb7172d411bdf36215cc02a32fe047fb0dd61c8470706\": container with ID starting with 4b632bdc07b0d0c2a22eb7172d411bdf36215cc02a32fe047fb0dd61c8470706 not found: ID does not exist" containerID="4b632bdc07b0d0c2a22eb7172d411bdf36215cc02a32fe047fb0dd61c8470706"
Jul 07 15:14:29.983185 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:29.983145    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b632bdc07b0d0c2a22eb7172d411bdf36215cc02a32fe047fb0dd61c8470706"} err="failed to get container status \"4b632bdc07b0d0c2a22eb7172d411bdf36215cc02a32fe047fb0dd61c8470706\": rpc error: code = NotFound desc = could not find container \"4b632bdc07b0d0c2a22eb7172d411bdf36215cc02a32fe047fb0dd61c8470706\": container with ID starting with 4b632bdc07b0d0c2a22eb7172d411bdf36215cc02a32fe047fb0dd61c8470706 not found: ID does not exist"
Jul 07 15:14:29.996100 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:29.996073    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b477bdf49-dln4b"]
Jul 07 15:14:30.001348 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:30.001324    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5b477bdf49-dln4b"]
Jul 07 15:14:30.778146 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:30.778115    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84e8f9e9-dd85-4b04-8594-1521b9a71765" path="/var/lib/kubelet/pods/84e8f9e9-dd85-4b04-8594-1521b9a71765/volumes"
Jul 07 15:14:30.832184 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:30.832152    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5b477bdf49-mslch"]
Jul 07 15:14:30.832559 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:30.832546    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84e8f9e9-dd85-4b04-8594-1521b9a71765" containerName="manager"
Jul 07 15:14:30.832602 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:30.832560    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="84e8f9e9-dd85-4b04-8594-1521b9a71765" containerName="manager"
Jul 07 15:14:30.832602 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:30.832575    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b35acac-02b1-4c22-a6ee-6bc321d7d0b0" containerName="authorino"
Jul 07 15:14:30.832602 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:30.832580    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b35acac-02b1-4c22-a6ee-6bc321d7d0b0" containerName="authorino"
Jul 07 15:14:30.832698 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:30.832650    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="84e8f9e9-dd85-4b04-8594-1521b9a71765" containerName="manager"
Jul 07 15:14:30.832698 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:30.832658    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b35acac-02b1-4c22-a6ee-6bc321d7d0b0" containerName="authorino"
Jul 07 15:14:30.837150 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:30.837129    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b477bdf49-mslch"
Jul 07 15:14:30.841370 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:30.841348    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dfpjj\""
Jul 07 15:14:30.841492 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:30.841353    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 07 15:14:30.841492 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:30.841353    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 07 15:14:30.844210 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:30.843943    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5339528d-0992-43a7-90bc-1dc2d587b56d-webhook-cert\") pod \"maas-controller-5b477bdf49-mslch\" (UID: \"5339528d-0992-43a7-90bc-1dc2d587b56d\") " pod="opendatahub/maas-controller-5b477bdf49-mslch"
Jul 07 15:14:30.844210 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:30.843981    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42c5b\" (UniqueName: \"kubernetes.io/projected/5339528d-0992-43a7-90bc-1dc2d587b56d-kube-api-access-42c5b\") pod \"maas-controller-5b477bdf49-mslch\" (UID: \"5339528d-0992-43a7-90bc-1dc2d587b56d\") " pod="opendatahub/maas-controller-5b477bdf49-mslch"
Jul 07 15:14:30.844945 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:30.844919    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b477bdf49-mslch"]
Jul 07 15:14:30.944936 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:30.944908    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5339528d-0992-43a7-90bc-1dc2d587b56d-webhook-cert\") pod \"maas-controller-5b477bdf49-mslch\" (UID: \"5339528d-0992-43a7-90bc-1dc2d587b56d\") " pod="opendatahub/maas-controller-5b477bdf49-mslch"
Jul 07 15:14:30.945101 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:30.944942    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42c5b\" (UniqueName: \"kubernetes.io/projected/5339528d-0992-43a7-90bc-1dc2d587b56d-kube-api-access-42c5b\") pod \"maas-controller-5b477bdf49-mslch\" (UID: \"5339528d-0992-43a7-90bc-1dc2d587b56d\") " pod="opendatahub/maas-controller-5b477bdf49-mslch"
Jul 07 15:14:30.947594 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:30.947557    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5339528d-0992-43a7-90bc-1dc2d587b56d-webhook-cert\") pod \"maas-controller-5b477bdf49-mslch\" (UID: \"5339528d-0992-43a7-90bc-1dc2d587b56d\") " pod="opendatahub/maas-controller-5b477bdf49-mslch"
Jul 07 15:14:30.954090 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:30.954067    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42c5b\" (UniqueName: \"kubernetes.io/projected/5339528d-0992-43a7-90bc-1dc2d587b56d-kube-api-access-42c5b\") pod \"maas-controller-5b477bdf49-mslch\" (UID: \"5339528d-0992-43a7-90bc-1dc2d587b56d\") " pod="opendatahub/maas-controller-5b477bdf49-mslch"
Jul 07 15:14:31.148461 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:31.148419    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b477bdf49-mslch"
Jul 07 15:14:31.293248 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:31.293221    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b477bdf49-mslch"]
Jul 07 15:14:31.295195 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:14:31.295165    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5339528d_0992_43a7_90bc_1dc2d587b56d.slice/crio-06bbc5f2bb1e8ad843e1d33a376493cb8b10e6d6f2c753f5b8785c5301f91cee WatchSource:0}: Error finding container 06bbc5f2bb1e8ad843e1d33a376493cb8b10e6d6f2c753f5b8785c5301f91cee: Status 404 returned error can't find the container with id 06bbc5f2bb1e8ad843e1d33a376493cb8b10e6d6f2c753f5b8785c5301f91cee
Jul 07 15:14:31.985054 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:31.985015    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b477bdf49-mslch" event={"ID":"5339528d-0992-43a7-90bc-1dc2d587b56d","Type":"ContainerStarted","Data":"d8c306e430b0e48abaffc2f205a1892fafdd505ef4c4944f95a95f45135cf5c7"}
Jul 07 15:14:31.985224 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:31.985072    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b477bdf49-mslch" event={"ID":"5339528d-0992-43a7-90bc-1dc2d587b56d","Type":"ContainerStarted","Data":"06bbc5f2bb1e8ad843e1d33a376493cb8b10e6d6f2c753f5b8785c5301f91cee"}
Jul 07 15:14:31.985224 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:31.985103    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5b477bdf49-mslch"
Jul 07 15:14:32.004587 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:32.004540    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5b477bdf49-mslch" podStartSLOduration=1.485618445 podStartE2EDuration="2.004521364s" totalImagesPullingTime="518.902919ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:14:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:14:32.002911448 +0000 UTC m=+839.799448199" watchObservedRunningTime="2026-07-07 15:14:32.004521364 +0000 UTC m=+839.801058091"
Jul 07 15:14:42.996410 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:14:42.996376    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5b477bdf49-mslch"
Jul 07 15:15:00.151617 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:15:00.151581    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29723955-lsxqp"]
Jul 07 15:15:00.155123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:15:00.155102    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723955-lsxqp"
Jul 07 15:15:00.165534 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:15:00.165504    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723955-lsxqp"]
Jul 07 15:15:00.329022 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:15:00.328985    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2l5l\" (UniqueName: \"kubernetes.io/projected/94d3966c-ea30-4fd4-98c4-b5e153421253-kube-api-access-t2l5l\") pod \"maas-api-key-cleanup-29723955-lsxqp\" (UID: \"94d3966c-ea30-4fd4-98c4-b5e153421253\") " pod="opendatahub/maas-api-key-cleanup-29723955-lsxqp"
Jul 07 15:15:00.430230 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:15:00.430150    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2l5l\" (UniqueName: \"kubernetes.io/projected/94d3966c-ea30-4fd4-98c4-b5e153421253-kube-api-access-t2l5l\") pod \"maas-api-key-cleanup-29723955-lsxqp\" (UID: \"94d3966c-ea30-4fd4-98c4-b5e153421253\") " pod="opendatahub/maas-api-key-cleanup-29723955-lsxqp"
Jul 07 15:15:00.443570 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:15:00.443532    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2l5l\" (UniqueName: \"kubernetes.io/projected/94d3966c-ea30-4fd4-98c4-b5e153421253-kube-api-access-t2l5l\") pod \"maas-api-key-cleanup-29723955-lsxqp\" (UID: \"94d3966c-ea30-4fd4-98c4-b5e153421253\") " pod="opendatahub/maas-api-key-cleanup-29723955-lsxqp"
Jul 07 15:15:00.467602 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:15:00.467558    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723955-lsxqp"
Jul 07 15:15:00.613536 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:15:00.613504    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723955-lsxqp"]
Jul 07 15:15:00.616915 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:15:00.616871    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94d3966c_ea30_4fd4_98c4_b5e153421253.slice/crio-93012038f3b9dc276604ceecdfcd0217d91e16896c2afb37ec455fd59b7b1e20 WatchSource:0}: Error finding container 93012038f3b9dc276604ceecdfcd0217d91e16896c2afb37ec455fd59b7b1e20: Status 404 returned error can't find the container with id 93012038f3b9dc276604ceecdfcd0217d91e16896c2afb37ec455fd59b7b1e20
Jul 07 15:15:01.136003 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:15:01.135964    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723955-lsxqp" event={"ID":"94d3966c-ea30-4fd4-98c4-b5e153421253","Type":"ContainerStarted","Data":"93012038f3b9dc276604ceecdfcd0217d91e16896c2afb37ec455fd59b7b1e20"}
Jul 07 15:15:03.148740 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:15:03.148700    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723955-lsxqp" event={"ID":"94d3966c-ea30-4fd4-98c4-b5e153421253","Type":"ContainerStarted","Data":"acaa2789dce7016379ff7f02b3a58bd76fd220551e0d0de6f783ed433ba0c23d"}
Jul 07 15:15:03.168281 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:15:03.168229    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29723955-lsxqp" podStartSLOduration=0.979315202 podStartE2EDuration="3.168213227s" totalImagesPullingTime="2.188898025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:15:03.166065228 +0000 UTC m=+870.962601957" watchObservedRunningTime="2026-07-07 15:15:03.168213227 +0000 UTC m=+870.964749945"
Jul 07 15:15:04.155459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:15:04.155352    2567 generic.go:356] "Generic (PLEG): container finished" podID="94d3966c-ea30-4fd4-98c4-b5e153421253" containerID="acaa2789dce7016379ff7f02b3a58bd76fd220551e0d0de6f783ed433ba0c23d" exitCode=0
Jul 07 15:15:04.155459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:15:04.155434    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723955-lsxqp" event={"ID":"94d3966c-ea30-4fd4-98c4-b5e153421253","Type":"ContainerDied","Data":"acaa2789dce7016379ff7f02b3a58bd76fd220551e0d0de6f783ed433ba0c23d"}
Jul 07 15:15:05.305054 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:15:05.305030    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723955-lsxqp"
Jul 07 15:15:05.478156 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:15:05.478063    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t2l5l\" (UniqueName: \"kubernetes.io/projected/94d3966c-ea30-4fd4-98c4-b5e153421253-kube-api-access-t2l5l\") pod \"94d3966c-ea30-4fd4-98c4-b5e153421253\" (UID: \"94d3966c-ea30-4fd4-98c4-b5e153421253\") "
Jul 07 15:15:05.480261 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:15:05.480230    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94d3966c-ea30-4fd4-98c4-b5e153421253-kube-api-access-t2l5l" (OuterVolumeSpecName: "kube-api-access-t2l5l") pod "94d3966c-ea30-4fd4-98c4-b5e153421253" (UID: "94d3966c-ea30-4fd4-98c4-b5e153421253"). InnerVolumeSpecName "kube-api-access-t2l5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:15:05.579699 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:15:05.579657    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2l5l\" (UniqueName: \"kubernetes.io/projected/94d3966c-ea30-4fd4-98c4-b5e153421253-kube-api-access-t2l5l\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:15:06.166616 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:15:06.166588    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723955-lsxqp"
Jul 07 15:15:06.166795 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:15:06.166615    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723955-lsxqp" event={"ID":"94d3966c-ea30-4fd4-98c4-b5e153421253","Type":"ContainerDied","Data":"93012038f3b9dc276604ceecdfcd0217d91e16896c2afb37ec455fd59b7b1e20"}
Jul 07 15:15:06.166795 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:15:06.166660    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="93012038f3b9dc276604ceecdfcd0217d91e16896c2afb37ec455fd59b7b1e20"
Jul 07 15:15:32.713459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:15:32.713425    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5r9rp_1c51cd93-b0ea-4aa1-8678-a4c833b2d37e/console-operator/1.log"
Jul 07 15:15:32.715971 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:15:32.715082    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5r9rp_1c51cd93-b0ea-4aa1-8678-a4c833b2d37e/console-operator/1.log"
Jul 07 15:19:20.503682 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:19:20.503643    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hp2v"]
Jul 07 15:19:20.504417 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:19:20.503853    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hp2v" podUID="8929b62d-4b20-4bd3-bfa0-5ed756f58425" containerName="manager" containerID="cri-o://00f776c488a87e94cdef526defbff66055a952fcfca8b87df2e4951e30916a0c" gracePeriod=10
Jul 07 15:19:23.531408 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:19:23.531375    2567 generic.go:356] "Generic (PLEG): container finished" podID="8929b62d-4b20-4bd3-bfa0-5ed756f58425" containerID="00f776c488a87e94cdef526defbff66055a952fcfca8b87df2e4951e30916a0c" exitCode=0
Jul 07 15:19:23.531778 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:19:23.531441    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hp2v" event={"ID":"8929b62d-4b20-4bd3-bfa0-5ed756f58425","Type":"ContainerDied","Data":"00f776c488a87e94cdef526defbff66055a952fcfca8b87df2e4951e30916a0c"}
Jul 07 15:19:23.663124 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:19:23.663096    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hp2v"
Jul 07 15:19:23.713977 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:19:23.713877    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8929b62d-4b20-4bd3-bfa0-5ed756f58425-extensions-socket-volume\") pod \"8929b62d-4b20-4bd3-bfa0-5ed756f58425\" (UID: \"8929b62d-4b20-4bd3-bfa0-5ed756f58425\") "
Jul 07 15:19:23.714151 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:19:23.713992    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z76fb\" (UniqueName: \"kubernetes.io/projected/8929b62d-4b20-4bd3-bfa0-5ed756f58425-kube-api-access-z76fb\") pod \"8929b62d-4b20-4bd3-bfa0-5ed756f58425\" (UID: \"8929b62d-4b20-4bd3-bfa0-5ed756f58425\") "
Jul 07 15:19:23.714332 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:19:23.714304    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8929b62d-4b20-4bd3-bfa0-5ed756f58425-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8929b62d-4b20-4bd3-bfa0-5ed756f58425" (UID: "8929b62d-4b20-4bd3-bfa0-5ed756f58425"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:19:23.716528 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:19:23.716497    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8929b62d-4b20-4bd3-bfa0-5ed756f58425-kube-api-access-z76fb" (OuterVolumeSpecName: "kube-api-access-z76fb") pod "8929b62d-4b20-4bd3-bfa0-5ed756f58425" (UID: "8929b62d-4b20-4bd3-bfa0-5ed756f58425"). InnerVolumeSpecName "kube-api-access-z76fb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:19:23.815425 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:19:23.815385    2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8929b62d-4b20-4bd3-bfa0-5ed756f58425-extensions-socket-volume\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:19:23.815425 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:19:23.815423    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z76fb\" (UniqueName: \"kubernetes.io/projected/8929b62d-4b20-4bd3-bfa0-5ed756f58425-kube-api-access-z76fb\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:19:24.538785 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:19:24.538750    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hp2v"
Jul 07 15:19:24.538785 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:19:24.538771    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hp2v" event={"ID":"8929b62d-4b20-4bd3-bfa0-5ed756f58425","Type":"ContainerDied","Data":"7163ab1c5d2ba08987c18a5712866028b3d32f8e11caaa9c414e53233e29327f"}
Jul 07 15:19:24.539273 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:19:24.538895    2567 scope.go:117] "RemoveContainer" containerID="00f776c488a87e94cdef526defbff66055a952fcfca8b87df2e4951e30916a0c"
Jul 07 15:19:24.563533 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:19:24.563502    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hp2v"]
Jul 07 15:19:24.566305 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:19:24.566283    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hp2v"]
Jul 07 15:19:24.778394 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:19:24.778361    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8929b62d-4b20-4bd3-bfa0-5ed756f58425" path="/var/lib/kubelet/pods/8929b62d-4b20-4bd3-bfa0-5ed756f58425/volumes"
Jul 07 15:20:21.685452 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:20:21.685417    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k22w"]
Jul 07 15:20:21.686015 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:20:21.685862    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94d3966c-ea30-4fd4-98c4-b5e153421253" containerName="cleanup"
Jul 07 15:20:21.686015 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:20:21.685875    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="94d3966c-ea30-4fd4-98c4-b5e153421253" containerName="cleanup"
Jul 07 15:20:21.686015 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:20:21.685898    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8929b62d-4b20-4bd3-bfa0-5ed756f58425" containerName="manager"
Jul 07 15:20:21.686015 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:20:21.685903    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8929b62d-4b20-4bd3-bfa0-5ed756f58425" containerName="manager"
Jul 07 15:20:21.686015 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:20:21.685973    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8929b62d-4b20-4bd3-bfa0-5ed756f58425" containerName="manager"
Jul 07 15:20:21.686015 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:20:21.685985    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="94d3966c-ea30-4fd4-98c4-b5e153421253" containerName="cleanup"
Jul 07 15:20:21.689221 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:20:21.689204    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k22w"
Jul 07 15:20:21.692261 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:20:21.692246    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-c48bh\""
Jul 07 15:20:21.707141 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:20:21.707116    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k22w"]
Jul 07 15:20:21.741613 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:20:21.741576    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1799722a-3db0-404b-b320-d3962ea95563-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7k22w\" (UID: \"1799722a-3db0-404b-b320-d3962ea95563\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k22w"
Jul 07 15:20:21.741613 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:20:21.741621    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bljb5\" (UniqueName: \"kubernetes.io/projected/1799722a-3db0-404b-b320-d3962ea95563-kube-api-access-bljb5\") pod \"kuadrant-operator-controller-manager-86878c769f-7k22w\" (UID: \"1799722a-3db0-404b-b320-d3962ea95563\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k22w"
Jul 07 15:20:21.842732 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:20:21.842696    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1799722a-3db0-404b-b320-d3962ea95563-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7k22w\" (UID: \"1799722a-3db0-404b-b320-d3962ea95563\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k22w"
Jul 07 15:20:21.842991 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:20:21.842764    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bljb5\" (UniqueName: \"kubernetes.io/projected/1799722a-3db0-404b-b320-d3962ea95563-kube-api-access-bljb5\") pod \"kuadrant-operator-controller-manager-86878c769f-7k22w\" (UID: \"1799722a-3db0-404b-b320-d3962ea95563\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k22w"
Jul 07 15:20:21.843177 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:20:21.843154    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1799722a-3db0-404b-b320-d3962ea95563-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7k22w\" (UID: \"1799722a-3db0-404b-b320-d3962ea95563\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k22w"
Jul 07 15:20:21.856619 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:20:21.856584    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bljb5\" (UniqueName: \"kubernetes.io/projected/1799722a-3db0-404b-b320-d3962ea95563-kube-api-access-bljb5\") pod \"kuadrant-operator-controller-manager-86878c769f-7k22w\" (UID: \"1799722a-3db0-404b-b320-d3962ea95563\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k22w"
Jul 07 15:20:21.999899 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:20:21.999806    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k22w"
Jul 07 15:20:22.158096 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:20:22.158069    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k22w"]
Jul 07 15:20:22.159130 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:20:22.159105    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1799722a_3db0_404b_b320_d3962ea95563.slice/crio-2401cfe4710f7d50e3561906964ef8524dcdf4bf24d8db7802dc6f8a68eeac95 WatchSource:0}: Error finding container 2401cfe4710f7d50e3561906964ef8524dcdf4bf24d8db7802dc6f8a68eeac95: Status 404 returned error can't find the container with id 2401cfe4710f7d50e3561906964ef8524dcdf4bf24d8db7802dc6f8a68eeac95
Jul 07 15:20:22.835134 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:20:22.835085    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k22w" event={"ID":"1799722a-3db0-404b-b320-d3962ea95563","Type":"ContainerStarted","Data":"39d75ba570690c2528e30a1156d0c3f12ed977d974875f5393bc9f5484dd564d"}
Jul 07 15:20:22.835564 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:20:22.835154    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k22w" event={"ID":"1799722a-3db0-404b-b320-d3962ea95563","Type":"ContainerStarted","Data":"2401cfe4710f7d50e3561906964ef8524dcdf4bf24d8db7802dc6f8a68eeac95"}
Jul 07 15:20:22.835564 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:20:22.835183    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k22w"
Jul 07 15:20:22.865237 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:20:22.865189    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k22w" podStartSLOduration=1.865173798 podStartE2EDuration="1.865173798s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:20:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:20:22.865011348 +0000 UTC m=+1190.661548078" watchObservedRunningTime="2026-07-07 15:20:22.865173798 +0000 UTC m=+1190.661710526"
Jul 07 15:20:32.745029 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:20:32.744913    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5r9rp_1c51cd93-b0ea-4aa1-8678-a4c833b2d37e/console-operator/1.log"
Jul 07 15:20:32.748917 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:20:32.748639    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5r9rp_1c51cd93-b0ea-4aa1-8678-a4c833b2d37e/console-operator/1.log"
Jul 07 15:20:33.843053 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:20:33.842967    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7k22w"
Jul 07 15:25:32.782608 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:25:32.782507    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5r9rp_1c51cd93-b0ea-4aa1-8678-a4c833b2d37e/console-operator/1.log"
Jul 07 15:25:32.786789 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:25:32.785948    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5r9rp_1c51cd93-b0ea-4aa1-8678-a4c833b2d37e/console-operator/1.log"
Jul 07 15:26:17.445520 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.445483    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"]
Jul 07 15:26:17.449202 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.449181    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.455478 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.455452    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-0a4a8aa5-openshift-default-dockercfg-m2ld9\""
Jul 07 15:26:17.483212 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.483181    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"]
Jul 07 15:26:17.488165 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.488142    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-workload-socket\") pod \"e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") " pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.488285 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.488175    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-credential-socket\") pod \"e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") " pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.488285 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.488195    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a5807a0-a209-4f80-b11d-36571619834e-istio-token\") pod \"e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") " pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.488285 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.488216    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt54p\" (UniqueName: \"kubernetes.io/projected/8a5807a0-a209-4f80-b11d-36571619834e-kube-api-access-dt54p\") pod \"e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") " pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.488511 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.488294    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a5807a0-a209-4f80-b11d-36571619834e-istio-podinfo\") pod \"e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") " pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.488511 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.488323    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-istio-envoy\") pod \"e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") " pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.488511 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.488401    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-workload-certs\") pod \"e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") " pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.488511 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.488448    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a5807a0-a209-4f80-b11d-36571619834e-istiod-ca-cert\") pod \"e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") " pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.488511 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.488482    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-istio-data\") pod \"e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") " pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.589604 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.589566    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-workload-socket\") pod \"e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") " pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.589604 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.589610    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-credential-socket\") pod \"e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") " pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.589928 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.589631    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a5807a0-a209-4f80-b11d-36571619834e-istio-token\") pod \"e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") " pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.589928 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.589757    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt54p\" (UniqueName: \"kubernetes.io/projected/8a5807a0-a209-4f80-b11d-36571619834e-kube-api-access-dt54p\") pod \"e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") " pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.589928 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.589869    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a5807a0-a209-4f80-b11d-36571619834e-istio-podinfo\") pod \"e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") " pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.590107 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.589929    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-istio-envoy\") pod \"e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") " pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.590107 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.589984    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-workload-certs\") pod \"e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") " pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.590107 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.590025    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a5807a0-a209-4f80-b11d-36571619834e-istiod-ca-cert\") pod \"e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") " pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.590107 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.590032    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-credential-socket\") pod \"e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") " pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.590107 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.590068    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-workload-socket\") pod \"e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") " pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.590400 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.590104    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-istio-data\") pod \"e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") " pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.590400 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.590321    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-workload-certs\") pod \"e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") " pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.590400 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.590362    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-istio-data\") pod \"e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") " pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.590688 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.590666    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a5807a0-a209-4f80-b11d-36571619834e-istiod-ca-cert\") pod \"e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") " pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.592258 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.592236    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-istio-envoy\") pod \"e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") " pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.592795 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.592777    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a5807a0-a209-4f80-b11d-36571619834e-istio-podinfo\") pod \"e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") " pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.599748 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.599726    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a5807a0-a209-4f80-b11d-36571619834e-istio-token\") pod \"e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") " pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.601096 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.601075    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt54p\" (UniqueName: \"kubernetes.io/projected/8a5807a0-a209-4f80-b11d-36571619834e-kube-api-access-dt54p\") pod \"e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") " pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.761136 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.761056    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:17.916444 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.916411    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"]
Jul 07 15:26:17.918137 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:26:17.918111    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a5807a0_a209_4f80_b11d_36571619834e.slice/crio-030d99f0bc583387ed0e236b7596d5c8a8be0e9a711b9fd8d2732bbf343b1790 WatchSource:0}: Error finding container 030d99f0bc583387ed0e236b7596d5c8a8be0e9a711b9fd8d2732bbf343b1790: Status 404 returned error can't find the container with id 030d99f0bc583387ed0e236b7596d5c8a8be0e9a711b9fd8d2732bbf343b1790
Jul 07 15:26:17.920405 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.920373    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:26:17.920515 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.920457    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:26:17.920515 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:17.920498    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:26:18.691359 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:18.691320    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8" event={"ID":"8a5807a0-a209-4f80-b11d-36571619834e","Type":"ContainerStarted","Data":"4d37f1ad3cb270ebaea21efb9dcae78081a92f89ca83d88d64fc887f4d77a184"}
Jul 07 15:26:18.691784 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:18.691392    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8" event={"ID":"8a5807a0-a209-4f80-b11d-36571619834e","Type":"ContainerStarted","Data":"030d99f0bc583387ed0e236b7596d5c8a8be0e9a711b9fd8d2732bbf343b1790"}
Jul 07 15:26:18.746431 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:18.746366    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8" podStartSLOduration=1.746343801 podStartE2EDuration="1.746343801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:26:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:26:18.745503732 +0000 UTC m=+1546.542040459" watchObservedRunningTime="2026-07-07 15:26:18.746343801 +0000 UTC m=+1546.542880530"
Jul 07 15:26:18.761647 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:18.761616    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:18.766984 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:18.766958    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:18.856812 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:18.856779    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5"]
Jul 07 15:26:18.861449 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:18.861423    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5"
Jul 07 15:26:18.864387 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:18.864365    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-0a4a8aa5-serving-cert\""
Jul 07 15:26:18.871101 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:18.871078    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5"]
Jul 07 15:26:18.901422 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:18.901383    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64c0f040-f740-425a-ad9e-512d394cba23-maas-api-tls\") pod \"maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5\" (UID: \"64c0f040-f740-425a-ad9e-512d394cba23\") " pod="opendatahub/maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5"
Jul 07 15:26:18.901597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:18.901569    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dll69\" (UniqueName: \"kubernetes.io/projected/64c0f040-f740-425a-ad9e-512d394cba23-kube-api-access-dll69\") pod \"maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5\" (UID: \"64c0f040-f740-425a-ad9e-512d394cba23\") " pod="opendatahub/maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5"
Jul 07 15:26:18.901673 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:18.901655    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64c0f040-f740-425a-ad9e-512d394cba23-openshift-service-ca\") pod \"maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5\" (UID: \"64c0f040-f740-425a-ad9e-512d394cba23\") " pod="opendatahub/maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5"
Jul 07 15:26:19.003135 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:19.003043    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dll69\" (UniqueName: \"kubernetes.io/projected/64c0f040-f740-425a-ad9e-512d394cba23-kube-api-access-dll69\") pod \"maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5\" (UID: \"64c0f040-f740-425a-ad9e-512d394cba23\") " pod="opendatahub/maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5"
Jul 07 15:26:19.003135 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:19.003107    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64c0f040-f740-425a-ad9e-512d394cba23-openshift-service-ca\") pod \"maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5\" (UID: \"64c0f040-f740-425a-ad9e-512d394cba23\") " pod="opendatahub/maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5"
Jul 07 15:26:19.003386 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:19.003147    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64c0f040-f740-425a-ad9e-512d394cba23-maas-api-tls\") pod \"maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5\" (UID: \"64c0f040-f740-425a-ad9e-512d394cba23\") " pod="opendatahub/maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5"
Jul 07 15:26:19.003386 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:26:19.003298    2567 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-0a4a8aa5-serving-cert: secret "maas-api-e2e-ait-0a4a8aa5-serving-cert" not found
Jul 07 15:26:19.003386 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:26:19.003377    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64c0f040-f740-425a-ad9e-512d394cba23-maas-api-tls podName:64c0f040-f740-425a-ad9e-512d394cba23 nodeName:}" failed. No retries permitted until 2026-07-07 15:26:19.503354532 +0000 UTC m=+1547.299891251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/64c0f040-f740-425a-ad9e-512d394cba23-maas-api-tls") pod "maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5" (UID: "64c0f040-f740-425a-ad9e-512d394cba23") : secret "maas-api-e2e-ait-0a4a8aa5-serving-cert" not found
Jul 07 15:26:19.003888 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:19.003852    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64c0f040-f740-425a-ad9e-512d394cba23-openshift-service-ca\") pod \"maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5\" (UID: \"64c0f040-f740-425a-ad9e-512d394cba23\") " pod="opendatahub/maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5"
Jul 07 15:26:19.015850 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:19.015807    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dll69\" (UniqueName: \"kubernetes.io/projected/64c0f040-f740-425a-ad9e-512d394cba23-kube-api-access-dll69\") pod \"maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5\" (UID: \"64c0f040-f740-425a-ad9e-512d394cba23\") " pod="opendatahub/maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5"
Jul 07 15:26:19.506808 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:19.506768    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64c0f040-f740-425a-ad9e-512d394cba23-maas-api-tls\") pod \"maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5\" (UID: \"64c0f040-f740-425a-ad9e-512d394cba23\") " pod="opendatahub/maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5"
Jul 07 15:26:19.509397 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:19.509371    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64c0f040-f740-425a-ad9e-512d394cba23-maas-api-tls\") pod \"maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5\" (UID: \"64c0f040-f740-425a-ad9e-512d394cba23\") " pod="opendatahub/maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5"
Jul 07 15:26:19.698079 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:19.698048    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:19.699009 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:19.698989    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:19.773710 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:19.773680    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5"
Jul 07 15:26:19.934662 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:19.934511    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5"]
Jul 07 15:26:19.937409 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:26:19.937374    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64c0f040_f740_425a_ad9e_512d394cba23.slice/crio-0d3e39e6b4b1f575361265ecf6017c4e47be370c6c0f68fd616ed203bdff5f62 WatchSource:0}: Error finding container 0d3e39e6b4b1f575361265ecf6017c4e47be370c6c0f68fd616ed203bdff5f62: Status 404 returned error can't find the container with id 0d3e39e6b4b1f575361265ecf6017c4e47be370c6c0f68fd616ed203bdff5f62
Jul 07 15:26:19.938913 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:19.938891    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 15:26:20.704099 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:20.704063    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5" event={"ID":"64c0f040-f740-425a-ad9e-512d394cba23","Type":"ContainerStarted","Data":"0d3e39e6b4b1f575361265ecf6017c4e47be370c6c0f68fd616ed203bdff5f62"}
Jul 07 15:26:21.711072 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:21.711038    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5" event={"ID":"64c0f040-f740-425a-ad9e-512d394cba23","Type":"ContainerStarted","Data":"14431596dbadc169bd26b3a0121378bb76552a54493603d10d09dd864e2b4d07"}
Jul 07 15:26:21.711498 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:21.711269    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5"
Jul 07 15:26:21.739774 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:21.739722    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5" podStartSLOduration=2.787119141 podStartE2EDuration="3.739706676s" totalImagesPullingTime="952.587535ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:26:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:26:21.737903397 +0000 UTC m=+1549.534440143" watchObservedRunningTime="2026-07-07 15:26:21.739706676 +0000 UTC m=+1549.536243404"
Jul 07 15:26:23.929482 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:23.929432    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"]
Jul 07 15:26:23.930155 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:23.929665    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8" podUID="8a5807a0-a209-4f80-b11d-36571619834e" containerName="istio-proxy" containerID="cri-o://4d37f1ad3cb270ebaea21efb9dcae78081a92f89ca83d88d64fc887f4d77a184" gracePeriod=30
Jul 07 15:26:24.306160 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:24.306122    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5"]
Jul 07 15:26:24.306369 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:24.306318    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5" podUID="64c0f040-f740-425a-ad9e-512d394cba23" containerName="maas-api" containerID="cri-o://14431596dbadc169bd26b3a0121378bb76552a54493603d10d09dd864e2b4d07" gracePeriod=30
Jul 07 15:26:24.605009 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:24.604985    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5"
Jul 07 15:26:24.662551 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:24.662511    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dll69\" (UniqueName: \"kubernetes.io/projected/64c0f040-f740-425a-ad9e-512d394cba23-kube-api-access-dll69\") pod \"64c0f040-f740-425a-ad9e-512d394cba23\" (UID: \"64c0f040-f740-425a-ad9e-512d394cba23\") "
Jul 07 15:26:24.662727 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:24.662568    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64c0f040-f740-425a-ad9e-512d394cba23-maas-api-tls\") pod \"64c0f040-f740-425a-ad9e-512d394cba23\" (UID: \"64c0f040-f740-425a-ad9e-512d394cba23\") "
Jul 07 15:26:24.662727 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:24.662607    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64c0f040-f740-425a-ad9e-512d394cba23-openshift-service-ca\") pod \"64c0f040-f740-425a-ad9e-512d394cba23\" (UID: \"64c0f040-f740-425a-ad9e-512d394cba23\") "
Jul 07 15:26:24.663088 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:24.663061    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64c0f040-f740-425a-ad9e-512d394cba23-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "64c0f040-f740-425a-ad9e-512d394cba23" (UID: "64c0f040-f740-425a-ad9e-512d394cba23"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:26:24.665029 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:24.664993    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64c0f040-f740-425a-ad9e-512d394cba23-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "64c0f040-f740-425a-ad9e-512d394cba23" (UID: "64c0f040-f740-425a-ad9e-512d394cba23"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:26:24.665029 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:24.665015    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64c0f040-f740-425a-ad9e-512d394cba23-kube-api-access-dll69" (OuterVolumeSpecName: "kube-api-access-dll69") pod "64c0f040-f740-425a-ad9e-512d394cba23" (UID: "64c0f040-f740-425a-ad9e-512d394cba23"). InnerVolumeSpecName "kube-api-access-dll69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:26:24.745429 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:24.745387    2567 generic.go:356] "Generic (PLEG): container finished" podID="64c0f040-f740-425a-ad9e-512d394cba23" containerID="14431596dbadc169bd26b3a0121378bb76552a54493603d10d09dd864e2b4d07" exitCode=0
Jul 07 15:26:24.745620 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:24.745437    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5"
Jul 07 15:26:24.745620 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:24.745488    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5" event={"ID":"64c0f040-f740-425a-ad9e-512d394cba23","Type":"ContainerDied","Data":"14431596dbadc169bd26b3a0121378bb76552a54493603d10d09dd864e2b4d07"}
Jul 07 15:26:24.745739 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:24.745618    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5" event={"ID":"64c0f040-f740-425a-ad9e-512d394cba23","Type":"ContainerDied","Data":"0d3e39e6b4b1f575361265ecf6017c4e47be370c6c0f68fd616ed203bdff5f62"}
Jul 07 15:26:24.745739 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:24.745654    2567 scope.go:117] "RemoveContainer" containerID="14431596dbadc169bd26b3a0121378bb76552a54493603d10d09dd864e2b4d07"
Jul 07 15:26:24.755945 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:24.755817    2567 scope.go:117] "RemoveContainer" containerID="14431596dbadc169bd26b3a0121378bb76552a54493603d10d09dd864e2b4d07"
Jul 07 15:26:24.756317 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:26:24.756289    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14431596dbadc169bd26b3a0121378bb76552a54493603d10d09dd864e2b4d07\": container with ID starting with 14431596dbadc169bd26b3a0121378bb76552a54493603d10d09dd864e2b4d07 not found: ID does not exist" containerID="14431596dbadc169bd26b3a0121378bb76552a54493603d10d09dd864e2b4d07"
Jul 07 15:26:24.756427 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:24.756329    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14431596dbadc169bd26b3a0121378bb76552a54493603d10d09dd864e2b4d07"} err="failed to get container status \"14431596dbadc169bd26b3a0121378bb76552a54493603d10d09dd864e2b4d07\": rpc error: code = NotFound desc = could not find container \"14431596dbadc169bd26b3a0121378bb76552a54493603d10d09dd864e2b4d07\": container with ID starting with 14431596dbadc169bd26b3a0121378bb76552a54493603d10d09dd864e2b4d07 not found: ID does not exist"
Jul 07 15:26:24.764007 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:24.763983    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dll69\" (UniqueName: \"kubernetes.io/projected/64c0f040-f740-425a-ad9e-512d394cba23-kube-api-access-dll69\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:26:24.764102 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:24.764009    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64c0f040-f740-425a-ad9e-512d394cba23-maas-api-tls\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:26:24.764102 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:24.764024    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64c0f040-f740-425a-ad9e-512d394cba23-openshift-service-ca\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:26:24.778177 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:24.778148    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5"]
Jul 07 15:26:24.778325 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:24.778187    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-0a4a8aa5-56f4655744-r7gm5"]
Jul 07 15:26:26.778700 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:26.778620    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64c0f040-f740-425a-ad9e-512d394cba23" path="/var/lib/kubelet/pods/64c0f040-f740-425a-ad9e-512d394cba23/volumes"
Jul 07 15:26:29.495426 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.495395    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:29.612827 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.612778    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-credential-socket\") pod \"8a5807a0-a209-4f80-b11d-36571619834e\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") "
Jul 07 15:26:29.612827 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.612871    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a5807a0-a209-4f80-b11d-36571619834e-istio-token\") pod \"8a5807a0-a209-4f80-b11d-36571619834e\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") "
Jul 07 15:26:29.613138 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.612891    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-workload-certs\") pod \"8a5807a0-a209-4f80-b11d-36571619834e\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") "
Jul 07 15:26:29.613138 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.612918    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-istio-envoy\") pod \"8a5807a0-a209-4f80-b11d-36571619834e\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") "
Jul 07 15:26:29.613138 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.612938    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a5807a0-a209-4f80-b11d-36571619834e-istio-podinfo\") pod \"8a5807a0-a209-4f80-b11d-36571619834e\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") "
Jul 07 15:26:29.613138 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.612990    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a5807a0-a209-4f80-b11d-36571619834e-istiod-ca-cert\") pod \"8a5807a0-a209-4f80-b11d-36571619834e\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") "
Jul 07 15:26:29.613138 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.613068    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-workload-socket\") pod \"8a5807a0-a209-4f80-b11d-36571619834e\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") "
Jul 07 15:26:29.613138 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.613071    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8a5807a0-a209-4f80-b11d-36571619834e" (UID: "8a5807a0-a209-4f80-b11d-36571619834e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:26:29.613138 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.613090    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dt54p\" (UniqueName: \"kubernetes.io/projected/8a5807a0-a209-4f80-b11d-36571619834e-kube-api-access-dt54p\") pod \"8a5807a0-a209-4f80-b11d-36571619834e\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") "
Jul 07 15:26:29.613138 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.613115    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-istio-data\") pod \"8a5807a0-a209-4f80-b11d-36571619834e\" (UID: \"8a5807a0-a209-4f80-b11d-36571619834e\") "
Jul 07 15:26:29.613572 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.613175    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8a5807a0-a209-4f80-b11d-36571619834e" (UID: "8a5807a0-a209-4f80-b11d-36571619834e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:26:29.613572 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.613327    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8a5807a0-a209-4f80-b11d-36571619834e" (UID: "8a5807a0-a209-4f80-b11d-36571619834e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:26:29.613572 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.613414    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a5807a0-a209-4f80-b11d-36571619834e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8a5807a0-a209-4f80-b11d-36571619834e" (UID: "8a5807a0-a209-4f80-b11d-36571619834e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:26:29.613572 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.613454    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-workload-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:26:29.613572 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.613502    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-credential-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:26:29.613572 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.613515    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-workload-certs\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:26:29.614286 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.614260    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-istio-data" (OuterVolumeSpecName: "istio-data") pod "8a5807a0-a209-4f80-b11d-36571619834e" (UID: "8a5807a0-a209-4f80-b11d-36571619834e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:26:29.615518 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.615457    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8a5807a0-a209-4f80-b11d-36571619834e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8a5807a0-a209-4f80-b11d-36571619834e" (UID: "8a5807a0-a209-4f80-b11d-36571619834e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 15:26:29.615712 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.615691    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a5807a0-a209-4f80-b11d-36571619834e-istio-token" (OuterVolumeSpecName: "istio-token") pod "8a5807a0-a209-4f80-b11d-36571619834e" (UID: "8a5807a0-a209-4f80-b11d-36571619834e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:26:29.615799 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.615773    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8a5807a0-a209-4f80-b11d-36571619834e" (UID: "8a5807a0-a209-4f80-b11d-36571619834e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:26:29.615887 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.615871    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a5807a0-a209-4f80-b11d-36571619834e-kube-api-access-dt54p" (OuterVolumeSpecName: "kube-api-access-dt54p") pod "8a5807a0-a209-4f80-b11d-36571619834e" (UID: "8a5807a0-a209-4f80-b11d-36571619834e"). InnerVolumeSpecName "kube-api-access-dt54p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:26:29.714512 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.714472    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a5807a0-a209-4f80-b11d-36571619834e-istiod-ca-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:26:29.714512 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.714510    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dt54p\" (UniqueName: \"kubernetes.io/projected/8a5807a0-a209-4f80-b11d-36571619834e-kube-api-access-dt54p\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:26:29.714512 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.714521    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-istio-data\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:26:29.714968 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.714531    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a5807a0-a209-4f80-b11d-36571619834e-istio-token\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:26:29.714968 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.714544    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a5807a0-a209-4f80-b11d-36571619834e-istio-envoy\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:26:29.714968 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.714557    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a5807a0-a209-4f80-b11d-36571619834e-istio-podinfo\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:26:29.777288 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.777252    2567 generic.go:356] "Generic (PLEG): container finished" podID="8a5807a0-a209-4f80-b11d-36571619834e" containerID="4d37f1ad3cb270ebaea21efb9dcae78081a92f89ca83d88d64fc887f4d77a184" exitCode=0
Jul 07 15:26:29.777464 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.777306    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"
Jul 07 15:26:29.777464 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.777330    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8" event={"ID":"8a5807a0-a209-4f80-b11d-36571619834e","Type":"ContainerDied","Data":"4d37f1ad3cb270ebaea21efb9dcae78081a92f89ca83d88d64fc887f4d77a184"}
Jul 07 15:26:29.777464 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.777387    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8" event={"ID":"8a5807a0-a209-4f80-b11d-36571619834e","Type":"ContainerDied","Data":"030d99f0bc583387ed0e236b7596d5c8a8be0e9a711b9fd8d2732bbf343b1790"}
Jul 07 15:26:29.777464 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.777417    2567 scope.go:117] "RemoveContainer" containerID="4d37f1ad3cb270ebaea21efb9dcae78081a92f89ca83d88d64fc887f4d77a184"
Jul 07 15:26:29.787093 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.787076    2567 scope.go:117] "RemoveContainer" containerID="4d37f1ad3cb270ebaea21efb9dcae78081a92f89ca83d88d64fc887f4d77a184"
Jul 07 15:26:29.787357 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:26:29.787335    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d37f1ad3cb270ebaea21efb9dcae78081a92f89ca83d88d64fc887f4d77a184\": container with ID starting with 4d37f1ad3cb270ebaea21efb9dcae78081a92f89ca83d88d64fc887f4d77a184 not found: ID does not exist" containerID="4d37f1ad3cb270ebaea21efb9dcae78081a92f89ca83d88d64fc887f4d77a184"
Jul 07 15:26:29.787406 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.787366    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d37f1ad3cb270ebaea21efb9dcae78081a92f89ca83d88d64fc887f4d77a184"} err="failed to get container status \"4d37f1ad3cb270ebaea21efb9dcae78081a92f89ca83d88d64fc887f4d77a184\": rpc error: code = NotFound desc = could not find container \"4d37f1ad3cb270ebaea21efb9dcae78081a92f89ca83d88d64fc887f4d77a184\": container with ID starting with 4d37f1ad3cb270ebaea21efb9dcae78081a92f89ca83d88d64fc887f4d77a184 not found: ID does not exist"
Jul 07 15:26:29.808719 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.808691    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"]
Jul 07 15:26:29.813648 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:29.813624    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-0a4a8aa5-openshift-default-c495d875c-cqhs8"]
Jul 07 15:26:30.561535 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.561496    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"]
Jul 07 15:26:30.567998 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.562722    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a5807a0-a209-4f80-b11d-36571619834e" containerName="istio-proxy"
Jul 07 15:26:30.567998 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.562747    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a5807a0-a209-4f80-b11d-36571619834e" containerName="istio-proxy"
Jul 07 15:26:30.567998 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.562791    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64c0f040-f740-425a-ad9e-512d394cba23" containerName="maas-api"
Jul 07 15:26:30.567998 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.562800    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="64c0f040-f740-425a-ad9e-512d394cba23" containerName="maas-api"
Jul 07 15:26:30.567998 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.563226    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a5807a0-a209-4f80-b11d-36571619834e" containerName="istio-proxy"
Jul 07 15:26:30.567998 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.563245    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="64c0f040-f740-425a-ad9e-512d394cba23" containerName="maas-api"
Jul 07 15:26:30.573675 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.573650    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:30.577636 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.577614    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-745c3a48-openshift-default-dockercfg-vbvq6\""
Jul 07 15:26:30.583120 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.583087    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"]
Jul 07 15:26:30.621999 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.621961    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-istio-envoy\") pod \"e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") " pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:30.622163 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.622000    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvvr2\" (UniqueName: \"kubernetes.io/projected/fc7445fa-2f22-4586-8818-2a29f79217fd-kube-api-access-kvvr2\") pod \"e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") " pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:30.622163 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.622091    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-credential-socket\") pod \"e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") " pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:30.622279 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.622159    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc7445fa-2f22-4586-8818-2a29f79217fd-istio-token\") pod \"e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") " pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:30.622279 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.622190    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-istio-data\") pod \"e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") " pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:30.622279 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.622220    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc7445fa-2f22-4586-8818-2a29f79217fd-istiod-ca-cert\") pod \"e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") " pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:30.622397 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.622282    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc7445fa-2f22-4586-8818-2a29f79217fd-istio-podinfo\") pod \"e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") " pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:30.622432 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.622387    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-workload-certs\") pod \"e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") " pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:30.622493 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.622475    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-workload-socket\") pod \"e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") " pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:30.723228 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.723193    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-istio-envoy\") pod \"e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") " pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:30.723228 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.723230    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvvr2\" (UniqueName: \"kubernetes.io/projected/fc7445fa-2f22-4586-8818-2a29f79217fd-kube-api-access-kvvr2\") pod \"e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") " pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:30.723476 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.723262    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-credential-socket\") pod \"e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") " pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:30.723476 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.723293    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc7445fa-2f22-4586-8818-2a29f79217fd-istio-token\") pod \"e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") " pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:30.723476 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.723318    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-istio-data\") pod \"e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") " pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:30.723476 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.723350    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc7445fa-2f22-4586-8818-2a29f79217fd-istiod-ca-cert\") pod \"e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") " pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:30.723476 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.723382    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc7445fa-2f22-4586-8818-2a29f79217fd-istio-podinfo\") pod \"e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") " pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:30.723476 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.723437    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-workload-certs\") pod \"e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") " pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:30.723789 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.723505    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-workload-socket\") pod \"e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") " pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:30.723789 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.723700    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-istio-data\") pod \"e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") " pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:30.723914 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.723869    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-workload-socket\") pod \"e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") " pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:30.723978 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.723952    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-credential-socket\") pod \"e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") " pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:30.724035 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.723989    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-workload-certs\") pod \"e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") " pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:30.724184 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.724166    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc7445fa-2f22-4586-8818-2a29f79217fd-istiod-ca-cert\") pod \"e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") " pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:30.725633 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.725602    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-istio-envoy\") pod \"e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") " pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:30.726275 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.726256    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc7445fa-2f22-4586-8818-2a29f79217fd-istio-podinfo\") pod \"e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") " pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:30.732253 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.732230    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc7445fa-2f22-4586-8818-2a29f79217fd-istio-token\") pod \"e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") " pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:30.732563 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.732544    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvvr2\" (UniqueName: \"kubernetes.io/projected/fc7445fa-2f22-4586-8818-2a29f79217fd-kube-api-access-kvvr2\") pod \"e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") " pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:30.777700 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.777666    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a5807a0-a209-4f80-b11d-36571619834e" path="/var/lib/kubelet/pods/8a5807a0-a209-4f80-b11d-36571619834e/volumes"
Jul 07 15:26:30.887313 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:30.887233    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:31.038716 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:31.038688    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"]
Jul 07 15:26:31.041548 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:26:31.041524    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc7445fa_2f22_4586_8818_2a29f79217fd.slice/crio-9766e7765a3ab2cfab26a2da0510b70b22cc020555e3af2c8abfda74d1259a85 WatchSource:0}: Error finding container 9766e7765a3ab2cfab26a2da0510b70b22cc020555e3af2c8abfda74d1259a85: Status 404 returned error can't find the container with id 9766e7765a3ab2cfab26a2da0510b70b22cc020555e3af2c8abfda74d1259a85
Jul 07 15:26:31.043997 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:31.043963    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:26:31.044090 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:31.044032    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:26:31.044090 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:31.044067    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:26:31.789173 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:31.789138    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr" event={"ID":"fc7445fa-2f22-4586-8818-2a29f79217fd","Type":"ContainerStarted","Data":"608224d2cab52936979d376e6a1d3514cce5c137484c22d51aeaa8c1d9a8869a"}
Jul 07 15:26:31.789675 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:31.789195    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr" event={"ID":"fc7445fa-2f22-4586-8818-2a29f79217fd","Type":"ContainerStarted","Data":"9766e7765a3ab2cfab26a2da0510b70b22cc020555e3af2c8abfda74d1259a85"}
Jul 07 15:26:31.822982 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:31.822908    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr" podStartSLOduration=1.822815554 podStartE2EDuration="1.822815554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:26:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:26:31.822367631 +0000 UTC m=+1559.618904389" watchObservedRunningTime="2026-07-07 15:26:31.822815554 +0000 UTC m=+1559.619352284"
Jul 07 15:26:31.888203 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:31.888171    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:31.937030 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:31.936992    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z"]
Jul 07 15:26:31.941194 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:31.941170    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z"
Jul 07 15:26:31.944698 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:31.944672    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-745c3a48-serving-cert\""
Jul 07 15:26:31.956121 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:31.956083    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z"]
Jul 07 15:26:32.035261 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:32.035221    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e55f7daf-907a-426d-814f-f1541de40daa-openshift-service-ca\") pod \"maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z\" (UID: \"e55f7daf-907a-426d-814f-f1541de40daa\") " pod="opendatahub/maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z"
Jul 07 15:26:32.035459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:32.035289    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e55f7daf-907a-426d-814f-f1541de40daa-maas-api-tls\") pod \"maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z\" (UID: \"e55f7daf-907a-426d-814f-f1541de40daa\") " pod="opendatahub/maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z"
Jul 07 15:26:32.035459 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:32.035345    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqcsk\" (UniqueName: \"kubernetes.io/projected/e55f7daf-907a-426d-814f-f1541de40daa-kube-api-access-zqcsk\") pod \"maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z\" (UID: \"e55f7daf-907a-426d-814f-f1541de40daa\") " pod="opendatahub/maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z"
Jul 07 15:26:32.136654 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:32.136607    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e55f7daf-907a-426d-814f-f1541de40daa-maas-api-tls\") pod \"maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z\" (UID: \"e55f7daf-907a-426d-814f-f1541de40daa\") " pod="opendatahub/maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z"
Jul 07 15:26:32.136873 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:32.136674    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqcsk\" (UniqueName: \"kubernetes.io/projected/e55f7daf-907a-426d-814f-f1541de40daa-kube-api-access-zqcsk\") pod \"maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z\" (UID: \"e55f7daf-907a-426d-814f-f1541de40daa\") " pod="opendatahub/maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z"
Jul 07 15:26:32.136873 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:32.136756    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e55f7daf-907a-426d-814f-f1541de40daa-openshift-service-ca\") pod \"maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z\" (UID: \"e55f7daf-907a-426d-814f-f1541de40daa\") " pod="opendatahub/maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z"
Jul 07 15:26:32.136873 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:26:32.136772    2567 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-745c3a48-serving-cert: secret "maas-api-e2e-ait-745c3a48-serving-cert" not found
Jul 07 15:26:32.137054 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:26:32.136882    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e55f7daf-907a-426d-814f-f1541de40daa-maas-api-tls podName:e55f7daf-907a-426d-814f-f1541de40daa nodeName:}" failed. No retries permitted until 2026-07-07 15:26:32.636856754 +0000 UTC m=+1560.433393465 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e55f7daf-907a-426d-814f-f1541de40daa-maas-api-tls") pod "maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z" (UID: "e55f7daf-907a-426d-814f-f1541de40daa") : secret "maas-api-e2e-ait-745c3a48-serving-cert" not found
Jul 07 15:26:32.137519 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:32.137494    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e55f7daf-907a-426d-814f-f1541de40daa-openshift-service-ca\") pod \"maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z\" (UID: \"e55f7daf-907a-426d-814f-f1541de40daa\") " pod="opendatahub/maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z"
Jul 07 15:26:32.147273 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:32.147246    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqcsk\" (UniqueName: \"kubernetes.io/projected/e55f7daf-907a-426d-814f-f1541de40daa-kube-api-access-zqcsk\") pod \"maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z\" (UID: \"e55f7daf-907a-426d-814f-f1541de40daa\") " pod="opendatahub/maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z"
Jul 07 15:26:32.641998 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:32.641965    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e55f7daf-907a-426d-814f-f1541de40daa-maas-api-tls\") pod \"maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z\" (UID: \"e55f7daf-907a-426d-814f-f1541de40daa\") " pod="opendatahub/maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z"
Jul 07 15:26:32.644815 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:32.644775    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e55f7daf-907a-426d-814f-f1541de40daa-maas-api-tls\") pod \"maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z\" (UID: \"e55f7daf-907a-426d-814f-f1541de40daa\") " pod="opendatahub/maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z"
Jul 07 15:26:32.855644 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:32.855618    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z"
Jul 07 15:26:32.893090 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:32.893030    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:33.017530 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:33.017499    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z"]
Jul 07 15:26:33.019424 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:26:33.019375    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode55f7daf_907a_426d_814f_f1541de40daa.slice/crio-d19741f1b5ae36458fe115d34dbe0cc6d8c5267879aeeba1cde6cd922b8b501b WatchSource:0}: Error finding container d19741f1b5ae36458fe115d34dbe0cc6d8c5267879aeeba1cde6cd922b8b501b: Status 404 returned error can't find the container with id d19741f1b5ae36458fe115d34dbe0cc6d8c5267879aeeba1cde6cd922b8b501b
Jul 07 15:26:33.801390 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:33.801274    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z" event={"ID":"e55f7daf-907a-426d-814f-f1541de40daa","Type":"ContainerStarted","Data":"71824804700a8c1dca38552bce17f45eecc3a94c9a43777d147a72b421194613"}
Jul 07 15:26:33.801390 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:33.801333    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z" event={"ID":"e55f7daf-907a-426d-814f-f1541de40daa","Type":"ContainerStarted","Data":"d19741f1b5ae36458fe115d34dbe0cc6d8c5267879aeeba1cde6cd922b8b501b"}
Jul 07 15:26:33.801721 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:33.801546    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z"
Jul 07 15:26:33.801721 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:33.801589    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:33.802517 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:33.802494    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:33.845028 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:33.844977    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z" podStartSLOduration=2.386467865 podStartE2EDuration="2.844960471s" totalImagesPullingTime="458.492606ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:26:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:26:33.842705959 +0000 UTC m=+1561.639242705" watchObservedRunningTime="2026-07-07 15:26:33.844960471 +0000 UTC m=+1561.641497198"
Jul 07 15:26:37.441882 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:37.441851    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z"]
Jul 07 15:26:37.442286 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:37.442039    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z" podUID="e55f7daf-907a-426d-814f-f1541de40daa" containerName="maas-api" containerID="cri-o://71824804700a8c1dca38552bce17f45eecc3a94c9a43777d147a72b421194613" gracePeriod=30
Jul 07 15:26:37.723937 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:37.723898    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z"
Jul 07 15:26:37.790991 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:37.790960    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqcsk\" (UniqueName: \"kubernetes.io/projected/e55f7daf-907a-426d-814f-f1541de40daa-kube-api-access-zqcsk\") pod \"e55f7daf-907a-426d-814f-f1541de40daa\" (UID: \"e55f7daf-907a-426d-814f-f1541de40daa\") "
Jul 07 15:26:37.791155 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:37.791086    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e55f7daf-907a-426d-814f-f1541de40daa-openshift-service-ca\") pod \"e55f7daf-907a-426d-814f-f1541de40daa\" (UID: \"e55f7daf-907a-426d-814f-f1541de40daa\") "
Jul 07 15:26:37.791155 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:37.791117    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e55f7daf-907a-426d-814f-f1541de40daa-maas-api-tls\") pod \"e55f7daf-907a-426d-814f-f1541de40daa\" (UID: \"e55f7daf-907a-426d-814f-f1541de40daa\") "
Jul 07 15:26:37.791406 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:37.791383    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e55f7daf-907a-426d-814f-f1541de40daa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e55f7daf-907a-426d-814f-f1541de40daa" (UID: "e55f7daf-907a-426d-814f-f1541de40daa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:26:37.793147 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:37.793124    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e55f7daf-907a-426d-814f-f1541de40daa-kube-api-access-zqcsk" (OuterVolumeSpecName: "kube-api-access-zqcsk") pod "e55f7daf-907a-426d-814f-f1541de40daa" (UID: "e55f7daf-907a-426d-814f-f1541de40daa"). InnerVolumeSpecName "kube-api-access-zqcsk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:26:37.793303 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:37.793288    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e55f7daf-907a-426d-814f-f1541de40daa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e55f7daf-907a-426d-814f-f1541de40daa" (UID: "e55f7daf-907a-426d-814f-f1541de40daa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:26:37.824116 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:37.824080    2567 generic.go:356] "Generic (PLEG): container finished" podID="e55f7daf-907a-426d-814f-f1541de40daa" containerID="71824804700a8c1dca38552bce17f45eecc3a94c9a43777d147a72b421194613" exitCode=0
Jul 07 15:26:37.824280 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:37.824127    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z"
Jul 07 15:26:37.824280 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:37.824130    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z" event={"ID":"e55f7daf-907a-426d-814f-f1541de40daa","Type":"ContainerDied","Data":"71824804700a8c1dca38552bce17f45eecc3a94c9a43777d147a72b421194613"}
Jul 07 15:26:37.824280 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:37.824254    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z" event={"ID":"e55f7daf-907a-426d-814f-f1541de40daa","Type":"ContainerDied","Data":"d19741f1b5ae36458fe115d34dbe0cc6d8c5267879aeeba1cde6cd922b8b501b"}
Jul 07 15:26:37.824387 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:37.824283    2567 scope.go:117] "RemoveContainer" containerID="71824804700a8c1dca38552bce17f45eecc3a94c9a43777d147a72b421194613"
Jul 07 15:26:37.834086 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:37.834062    2567 scope.go:117] "RemoveContainer" containerID="71824804700a8c1dca38552bce17f45eecc3a94c9a43777d147a72b421194613"
Jul 07 15:26:37.834377 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:26:37.834355    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71824804700a8c1dca38552bce17f45eecc3a94c9a43777d147a72b421194613\": container with ID starting with 71824804700a8c1dca38552bce17f45eecc3a94c9a43777d147a72b421194613 not found: ID does not exist" containerID="71824804700a8c1dca38552bce17f45eecc3a94c9a43777d147a72b421194613"
Jul 07 15:26:37.834464 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:37.834386    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71824804700a8c1dca38552bce17f45eecc3a94c9a43777d147a72b421194613"} err="failed to get container status \"71824804700a8c1dca38552bce17f45eecc3a94c9a43777d147a72b421194613\": rpc error: code = NotFound desc = could not find container \"71824804700a8c1dca38552bce17f45eecc3a94c9a43777d147a72b421194613\": container with ID starting with 71824804700a8c1dca38552bce17f45eecc3a94c9a43777d147a72b421194613 not found: ID does not exist"
Jul 07 15:26:37.848752 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:37.848720    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z"]
Jul 07 15:26:37.854117 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:37.854090    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-745c3a48-6f466cf494-b6x8z"]
Jul 07 15:26:37.892805 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:37.892769    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqcsk\" (UniqueName: \"kubernetes.io/projected/e55f7daf-907a-426d-814f-f1541de40daa-kube-api-access-zqcsk\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:26:37.892976 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:37.892877    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e55f7daf-907a-426d-814f-f1541de40daa-openshift-service-ca\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:26:37.893039 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:37.893018    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e55f7daf-907a-426d-814f-f1541de40daa-maas-api-tls\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:26:38.778122 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:38.778091    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e55f7daf-907a-426d-814f-f1541de40daa" path="/var/lib/kubelet/pods/e55f7daf-907a-426d-814f-f1541de40daa/volumes"
Jul 07 15:26:43.135110 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:43.135036    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"]
Jul 07 15:26:43.135642 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:43.135401    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr" podUID="fc7445fa-2f22-4586-8818-2a29f79217fd" containerName="istio-proxy" containerID="cri-o://608224d2cab52936979d376e6a1d3514cce5c137484c22d51aeaa8c1d9a8869a" gracePeriod=30
Jul 07 15:26:46.604146 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.604119    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:46.781015 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.780931    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvvr2\" (UniqueName: \"kubernetes.io/projected/fc7445fa-2f22-4586-8818-2a29f79217fd-kube-api-access-kvvr2\") pod \"fc7445fa-2f22-4586-8818-2a29f79217fd\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") "
Jul 07 15:26:46.781015 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.780963    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc7445fa-2f22-4586-8818-2a29f79217fd-istiod-ca-cert\") pod \"fc7445fa-2f22-4586-8818-2a29f79217fd\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") "
Jul 07 15:26:46.781015 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.780981    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc7445fa-2f22-4586-8818-2a29f79217fd-istio-podinfo\") pod \"fc7445fa-2f22-4586-8818-2a29f79217fd\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") "
Jul 07 15:26:46.781015 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.781008    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc7445fa-2f22-4586-8818-2a29f79217fd-istio-token\") pod \"fc7445fa-2f22-4586-8818-2a29f79217fd\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") "
Jul 07 15:26:46.781345 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.781127    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-workload-certs\") pod \"fc7445fa-2f22-4586-8818-2a29f79217fd\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") "
Jul 07 15:26:46.781345 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.781179    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-istio-envoy\") pod \"fc7445fa-2f22-4586-8818-2a29f79217fd\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") "
Jul 07 15:26:46.781345 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.781278    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-credential-socket\") pod \"fc7445fa-2f22-4586-8818-2a29f79217fd\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") "
Jul 07 15:26:46.781345 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.781318    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-workload-socket\") pod \"fc7445fa-2f22-4586-8818-2a29f79217fd\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") "
Jul 07 15:26:46.781556 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.781360    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc7445fa-2f22-4586-8818-2a29f79217fd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fc7445fa-2f22-4586-8818-2a29f79217fd" (UID: "fc7445fa-2f22-4586-8818-2a29f79217fd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:26:46.781556 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.781381    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-istio-data\") pod \"fc7445fa-2f22-4586-8818-2a29f79217fd\" (UID: \"fc7445fa-2f22-4586-8818-2a29f79217fd\") "
Jul 07 15:26:46.781556 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.781507    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fc7445fa-2f22-4586-8818-2a29f79217fd" (UID: "fc7445fa-2f22-4586-8818-2a29f79217fd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:26:46.781556 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.781526    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fc7445fa-2f22-4586-8818-2a29f79217fd" (UID: "fc7445fa-2f22-4586-8818-2a29f79217fd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:26:46.781956 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.781729    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-credential-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:26:46.781956 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.781731    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fc7445fa-2f22-4586-8818-2a29f79217fd" (UID: "fc7445fa-2f22-4586-8818-2a29f79217fd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:26:46.781956 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.781752    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-workload-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:26:46.781956 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.781768    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc7445fa-2f22-4586-8818-2a29f79217fd-istiod-ca-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:26:46.782565 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.782536    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-istio-data" (OuterVolumeSpecName: "istio-data") pod "fc7445fa-2f22-4586-8818-2a29f79217fd" (UID: "fc7445fa-2f22-4586-8818-2a29f79217fd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:26:46.783477 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.783454    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc7445fa-2f22-4586-8818-2a29f79217fd-kube-api-access-kvvr2" (OuterVolumeSpecName: "kube-api-access-kvvr2") pod "fc7445fa-2f22-4586-8818-2a29f79217fd" (UID: "fc7445fa-2f22-4586-8818-2a29f79217fd"). InnerVolumeSpecName "kube-api-access-kvvr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:26:46.783563 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.783502    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fc7445fa-2f22-4586-8818-2a29f79217fd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fc7445fa-2f22-4586-8818-2a29f79217fd" (UID: "fc7445fa-2f22-4586-8818-2a29f79217fd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 15:26:46.783697 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.783670    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fc7445fa-2f22-4586-8818-2a29f79217fd" (UID: "fc7445fa-2f22-4586-8818-2a29f79217fd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:26:46.783793 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.783677    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc7445fa-2f22-4586-8818-2a29f79217fd-istio-token" (OuterVolumeSpecName: "istio-token") pod "fc7445fa-2f22-4586-8818-2a29f79217fd" (UID: "fc7445fa-2f22-4586-8818-2a29f79217fd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:26:46.883083 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.883045    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-istio-data\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:26:46.883083 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.883069    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvvr2\" (UniqueName: \"kubernetes.io/projected/fc7445fa-2f22-4586-8818-2a29f79217fd-kube-api-access-kvvr2\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:26:46.883083 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.883079    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc7445fa-2f22-4586-8818-2a29f79217fd-istio-podinfo\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:26:46.883083 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.883089    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc7445fa-2f22-4586-8818-2a29f79217fd-istio-token\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:26:46.883430 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.883103    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-workload-certs\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:26:46.883430 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.883111    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc7445fa-2f22-4586-8818-2a29f79217fd-istio-envoy\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:26:46.884003 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.883972    2567 generic.go:356] "Generic (PLEG): container finished" podID="fc7445fa-2f22-4586-8818-2a29f79217fd" containerID="608224d2cab52936979d376e6a1d3514cce5c137484c22d51aeaa8c1d9a8869a" exitCode=0
Jul 07 15:26:46.884123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.884020    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"
Jul 07 15:26:46.884123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.884049    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr" event={"ID":"fc7445fa-2f22-4586-8818-2a29f79217fd","Type":"ContainerDied","Data":"608224d2cab52936979d376e6a1d3514cce5c137484c22d51aeaa8c1d9a8869a"}
Jul 07 15:26:46.884123 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.884107    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr" event={"ID":"fc7445fa-2f22-4586-8818-2a29f79217fd","Type":"ContainerDied","Data":"9766e7765a3ab2cfab26a2da0510b70b22cc020555e3af2c8abfda74d1259a85"}
Jul 07 15:26:46.884243 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.884137    2567 scope.go:117] "RemoveContainer" containerID="608224d2cab52936979d376e6a1d3514cce5c137484c22d51aeaa8c1d9a8869a"
Jul 07 15:26:46.893810 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.893791    2567 scope.go:117] "RemoveContainer" containerID="608224d2cab52936979d376e6a1d3514cce5c137484c22d51aeaa8c1d9a8869a"
Jul 07 15:26:46.894158 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:26:46.894140    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"608224d2cab52936979d376e6a1d3514cce5c137484c22d51aeaa8c1d9a8869a\": container with ID starting with 608224d2cab52936979d376e6a1d3514cce5c137484c22d51aeaa8c1d9a8869a not found: ID does not exist" containerID="608224d2cab52936979d376e6a1d3514cce5c137484c22d51aeaa8c1d9a8869a"
Jul 07 15:26:46.894209 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.894164    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"608224d2cab52936979d376e6a1d3514cce5c137484c22d51aeaa8c1d9a8869a"} err="failed to get container status \"608224d2cab52936979d376e6a1d3514cce5c137484c22d51aeaa8c1d9a8869a\": rpc error: code = NotFound desc = could not find container \"608224d2cab52936979d376e6a1d3514cce5c137484c22d51aeaa8c1d9a8869a\": container with ID starting with 608224d2cab52936979d376e6a1d3514cce5c137484c22d51aeaa8c1d9a8869a not found: ID does not exist"
Jul 07 15:26:46.909840 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.909808    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"]
Jul 07 15:26:46.920648 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:46.920623    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-745c3a48-openshift-default-59d44c9477-2zwvr"]
Jul 07 15:26:48.779511 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:48.779475    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc7445fa-2f22-4586-8818-2a29f79217fd" path="/var/lib/kubelet/pods/fc7445fa-2f22-4586-8818-2a29f79217fd/volumes"
Jul 07 15:26:50.024383 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.024352    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"]
Jul 07 15:26:50.025127 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.025105    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc7445fa-2f22-4586-8818-2a29f79217fd" containerName="istio-proxy"
Jul 07 15:26:50.025243 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.025132    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc7445fa-2f22-4586-8818-2a29f79217fd" containerName="istio-proxy"
Jul 07 15:26:50.025243 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.025147    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e55f7daf-907a-426d-814f-f1541de40daa" containerName="maas-api"
Jul 07 15:26:50.025243 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.025156    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e55f7daf-907a-426d-814f-f1541de40daa" containerName="maas-api"
Jul 07 15:26:50.025368 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.025277    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e55f7daf-907a-426d-814f-f1541de40daa" containerName="maas-api"
Jul 07 15:26:50.025368 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.025297    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc7445fa-2f22-4586-8818-2a29f79217fd" containerName="istio-proxy"
Jul 07 15:26:50.030485 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.030461    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.034119 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.034089    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-2d4432df-openshift-default-dockercfg-f8kzv\""
Jul 07 15:26:50.047497 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.047464    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"]
Jul 07 15:26:50.214259 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.214201    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh557\" (UniqueName: \"kubernetes.io/projected/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-kube-api-access-hh557\") pod \"e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") " pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.214259 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.214266    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istio-token\") pod \"e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") " pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.214483 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.214345    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istio-data\") pod \"e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") " pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.214483 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.214404    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-workload-socket\") pod \"e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") " pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.214483 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.214428    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istiod-ca-cert\") pod \"e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") " pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.214585 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.214497    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-credential-socket\") pod \"e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") " pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.214585 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.214531    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istio-podinfo\") pod \"e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") " pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.214585 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.214569    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istio-envoy\") pod \"e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") " pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.214682 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.214623    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-workload-certs\") pod \"e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") " pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.315224 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.315179    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istio-data\") pod \"e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") " pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.315403 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.315242    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-workload-socket\") pod \"e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") " pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.315403 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.315270    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istiod-ca-cert\") pod \"e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") " pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.315403 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.315318    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-credential-socket\") pod \"e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") " pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.315403 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.315350    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istio-podinfo\") pod \"e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") " pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.315403 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.315397    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istio-envoy\") pod \"e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") " pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.315686 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.315450    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-workload-certs\") pod \"e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") " pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.315686 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.315489    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh557\" (UniqueName: \"kubernetes.io/projected/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-kube-api-access-hh557\") pod \"e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") " pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.315686 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.315522    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istio-token\") pod \"e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") " pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.315686 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.315643    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istio-data\") pod \"e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") " pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.316229 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.315909    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-credential-socket\") pod \"e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") " pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.316229 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.316074    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-workload-socket\") pod \"e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") " pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.316229 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.316108    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istiod-ca-cert\") pod \"e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") " pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.316229 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.316160    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-workload-certs\") pod \"e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") " pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.318067 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.318045    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istio-envoy\") pod \"e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") " pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.318319 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.318299    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istio-podinfo\") pod \"e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") " pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.326470 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.326447    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istio-token\") pod \"e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") " pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.326627 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.326606    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh557\" (UniqueName: \"kubernetes.io/projected/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-kube-api-access-hh557\") pod \"e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") " pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.341534 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.341508    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:50.504904 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.504870    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"]
Jul 07 15:26:50.505884 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:26:50.505825    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode79b9eaf_ebc3_42c2_98c8_176db1c380d6.slice/crio-f5fec833d1b7b289c44bd98d1b9a843b09e7a4fbd3fff448de68f3efd752cffc WatchSource:0}: Error finding container f5fec833d1b7b289c44bd98d1b9a843b09e7a4fbd3fff448de68f3efd752cffc: Status 404 returned error can't find the container with id f5fec833d1b7b289c44bd98d1b9a843b09e7a4fbd3fff448de68f3efd752cffc
Jul 07 15:26:50.508475 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.508445    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:26:50.508554 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.508510    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:26:50.508554 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.508545    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:26:50.907162 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.907128    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj" event={"ID":"e79b9eaf-ebc3-42c2-98c8-176db1c380d6","Type":"ContainerStarted","Data":"75c83be78d740842b6d536edc2e2a79591b9b6fa41d50827236d068a7ffd5178"}
Jul 07 15:26:50.907367 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.907181    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj" event={"ID":"e79b9eaf-ebc3-42c2-98c8-176db1c380d6","Type":"ContainerStarted","Data":"f5fec833d1b7b289c44bd98d1b9a843b09e7a4fbd3fff448de68f3efd752cffc"}
Jul 07 15:26:50.945363 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:50.945305    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj" podStartSLOduration=1.945283004 podStartE2EDuration="1.945283004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:26:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:26:50.94387014 +0000 UTC m=+1578.740406895" watchObservedRunningTime="2026-07-07 15:26:50.945283004 +0000 UTC m=+1578.741819732"
Jul 07 15:26:51.274962 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:51.274874    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg"]
Jul 07 15:26:51.286027 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:51.285992    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg"
Jul 07 15:26:51.288605 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:51.288576    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg"]
Jul 07 15:26:51.289436 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:51.289408    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-2d4432df-serving-cert\""
Jul 07 15:26:51.342692 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:51.342652    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:51.347487 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:51.347458    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:51.428218 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:51.428178    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a774a0b7-cf1a-483b-a11f-c18d96b762f6-openshift-service-ca\") pod \"maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg\" (UID: \"a774a0b7-cf1a-483b-a11f-c18d96b762f6\") " pod="opendatahub/maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg"
Jul 07 15:26:51.428382 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:51.428257    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8c8k\" (UniqueName: \"kubernetes.io/projected/a774a0b7-cf1a-483b-a11f-c18d96b762f6-kube-api-access-z8c8k\") pod \"maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg\" (UID: \"a774a0b7-cf1a-483b-a11f-c18d96b762f6\") " pod="opendatahub/maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg"
Jul 07 15:26:51.428382 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:51.428355    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a774a0b7-cf1a-483b-a11f-c18d96b762f6-maas-api-tls\") pod \"maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg\" (UID: \"a774a0b7-cf1a-483b-a11f-c18d96b762f6\") " pod="opendatahub/maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg"
Jul 07 15:26:51.529301 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:51.529209    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a774a0b7-cf1a-483b-a11f-c18d96b762f6-openshift-service-ca\") pod \"maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg\" (UID: \"a774a0b7-cf1a-483b-a11f-c18d96b762f6\") " pod="opendatahub/maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg"
Jul 07 15:26:51.529301 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:51.529272    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8c8k\" (UniqueName: \"kubernetes.io/projected/a774a0b7-cf1a-483b-a11f-c18d96b762f6-kube-api-access-z8c8k\") pod \"maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg\" (UID: \"a774a0b7-cf1a-483b-a11f-c18d96b762f6\") " pod="opendatahub/maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg"
Jul 07 15:26:51.529506 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:51.529312    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a774a0b7-cf1a-483b-a11f-c18d96b762f6-maas-api-tls\") pod \"maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg\" (UID: \"a774a0b7-cf1a-483b-a11f-c18d96b762f6\") " pod="opendatahub/maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg"
Jul 07 15:26:51.529506 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:26:51.529419    2567 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-2d4432df-serving-cert: secret "maas-api-e2e-derive-2d4432df-serving-cert" not found
Jul 07 15:26:51.529506 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:26:51.529475    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a774a0b7-cf1a-483b-a11f-c18d96b762f6-maas-api-tls podName:a774a0b7-cf1a-483b-a11f-c18d96b762f6 nodeName:}" failed. No retries permitted until 2026-07-07 15:26:52.029458302 +0000 UTC m=+1579.825995008 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a774a0b7-cf1a-483b-a11f-c18d96b762f6-maas-api-tls") pod "maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg" (UID: "a774a0b7-cf1a-483b-a11f-c18d96b762f6") : secret "maas-api-e2e-derive-2d4432df-serving-cert" not found
Jul 07 15:26:51.529953 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:51.529932    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a774a0b7-cf1a-483b-a11f-c18d96b762f6-openshift-service-ca\") pod \"maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg\" (UID: \"a774a0b7-cf1a-483b-a11f-c18d96b762f6\") " pod="opendatahub/maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg"
Jul 07 15:26:51.544421 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:51.544391    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8c8k\" (UniqueName: \"kubernetes.io/projected/a774a0b7-cf1a-483b-a11f-c18d96b762f6-kube-api-access-z8c8k\") pod \"maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg\" (UID: \"a774a0b7-cf1a-483b-a11f-c18d96b762f6\") " pod="opendatahub/maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg"
Jul 07 15:26:51.912440 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:51.912405    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:51.913462 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:51.913442    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:26:52.035103 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:52.035069    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a774a0b7-cf1a-483b-a11f-c18d96b762f6-maas-api-tls\") pod \"maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg\" (UID: \"a774a0b7-cf1a-483b-a11f-c18d96b762f6\") " pod="opendatahub/maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg"
Jul 07 15:26:52.038048 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:52.038017    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a774a0b7-cf1a-483b-a11f-c18d96b762f6-maas-api-tls\") pod \"maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg\" (UID: \"a774a0b7-cf1a-483b-a11f-c18d96b762f6\") " pod="opendatahub/maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg"
Jul 07 15:26:52.198793 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:52.198688    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg"
Jul 07 15:26:52.359098 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:52.359073    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg"]
Jul 07 15:26:52.360639 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:26:52.360614    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda774a0b7_cf1a_483b_a11f_c18d96b762f6.slice/crio-de619cd5def7245c4f320bad610f339fe877288a2fe260da8974afc59c076ed3 WatchSource:0}: Error finding container de619cd5def7245c4f320bad610f339fe877288a2fe260da8974afc59c076ed3: Status 404 returned error can't find the container with id de619cd5def7245c4f320bad610f339fe877288a2fe260da8974afc59c076ed3
Jul 07 15:26:52.923089 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:52.923053    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg" event={"ID":"a774a0b7-cf1a-483b-a11f-c18d96b762f6","Type":"ContainerStarted","Data":"14f05f2115206b903181d65c6b3054e65b93809624cb98dcdeaf5b5b3498c5a3"}
Jul 07 15:26:52.923257 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:52.923121    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg" event={"ID":"a774a0b7-cf1a-483b-a11f-c18d96b762f6","Type":"ContainerStarted","Data":"de619cd5def7245c4f320bad610f339fe877288a2fe260da8974afc59c076ed3"}
Jul 07 15:26:52.952249 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:52.952144    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg" podStartSLOduration=1.638398634 podStartE2EDuration="1.95212962s" totalImagesPullingTime="313.730986ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:26:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:26:52.951718197 +0000 UTC m=+1580.748254926" watchObservedRunningTime="2026-07-07 15:26:52.95212962 +0000 UTC m=+1580.748666347"
Jul 07 15:26:53.928748 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:53.928711    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg"
Jul 07 15:26:55.840663 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:55.840623    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"]
Jul 07 15:26:55.841581 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:55.841545    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj" podUID="e79b9eaf-ebc3-42c2-98c8-176db1c380d6" containerName="istio-proxy" containerID="cri-o://75c83be78d740842b6d536edc2e2a79591b9b6fa41d50827236d068a7ffd5178" gracePeriod=30
Jul 07 15:26:56.226150 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:56.224617    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg"]
Jul 07 15:26:56.230283 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:56.229952    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg" podUID="a774a0b7-cf1a-483b-a11f-c18d96b762f6" containerName="maas-api" containerID="cri-o://14f05f2115206b903181d65c6b3054e65b93809624cb98dcdeaf5b5b3498c5a3" gracePeriod=30
Jul 07 15:26:56.545801 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:56.545770    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg"
Jul 07 15:26:56.576995 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:56.576961    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a774a0b7-cf1a-483b-a11f-c18d96b762f6-openshift-service-ca\") pod \"a774a0b7-cf1a-483b-a11f-c18d96b762f6\" (UID: \"a774a0b7-cf1a-483b-a11f-c18d96b762f6\") "
Jul 07 15:26:56.577188 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:56.577065    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8c8k\" (UniqueName: \"kubernetes.io/projected/a774a0b7-cf1a-483b-a11f-c18d96b762f6-kube-api-access-z8c8k\") pod \"a774a0b7-cf1a-483b-a11f-c18d96b762f6\" (UID: \"a774a0b7-cf1a-483b-a11f-c18d96b762f6\") "
Jul 07 15:26:56.577188 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:56.577085    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a774a0b7-cf1a-483b-a11f-c18d96b762f6-maas-api-tls\") pod \"a774a0b7-cf1a-483b-a11f-c18d96b762f6\" (UID: \"a774a0b7-cf1a-483b-a11f-c18d96b762f6\") "
Jul 07 15:26:56.577341 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:56.577309    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a774a0b7-cf1a-483b-a11f-c18d96b762f6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a774a0b7-cf1a-483b-a11f-c18d96b762f6" (UID: "a774a0b7-cf1a-483b-a11f-c18d96b762f6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:26:56.579375 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:56.579344    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a774a0b7-cf1a-483b-a11f-c18d96b762f6-kube-api-access-z8c8k" (OuterVolumeSpecName: "kube-api-access-z8c8k") pod "a774a0b7-cf1a-483b-a11f-c18d96b762f6" (UID: "a774a0b7-cf1a-483b-a11f-c18d96b762f6"). InnerVolumeSpecName "kube-api-access-z8c8k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:26:56.579489 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:56.579384    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a774a0b7-cf1a-483b-a11f-c18d96b762f6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a774a0b7-cf1a-483b-a11f-c18d96b762f6" (UID: "a774a0b7-cf1a-483b-a11f-c18d96b762f6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:26:56.678016 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:56.677979    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a774a0b7-cf1a-483b-a11f-c18d96b762f6-openshift-service-ca\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:26:56.678016 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:56.678008    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8c8k\" (UniqueName: \"kubernetes.io/projected/a774a0b7-cf1a-483b-a11f-c18d96b762f6-kube-api-access-z8c8k\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:26:56.678016 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:56.678018    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a774a0b7-cf1a-483b-a11f-c18d96b762f6-maas-api-tls\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:26:56.952955 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:56.952915    2567 generic.go:356] "Generic (PLEG): container finished" podID="a774a0b7-cf1a-483b-a11f-c18d96b762f6" containerID="14f05f2115206b903181d65c6b3054e65b93809624cb98dcdeaf5b5b3498c5a3" exitCode=0
Jul 07 15:26:56.953392 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:56.952978    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg"
Jul 07 15:26:56.953392 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:56.953001    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg" event={"ID":"a774a0b7-cf1a-483b-a11f-c18d96b762f6","Type":"ContainerDied","Data":"14f05f2115206b903181d65c6b3054e65b93809624cb98dcdeaf5b5b3498c5a3"}
Jul 07 15:26:56.953392 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:56.953063    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg" event={"ID":"a774a0b7-cf1a-483b-a11f-c18d96b762f6","Type":"ContainerDied","Data":"de619cd5def7245c4f320bad610f339fe877288a2fe260da8974afc59c076ed3"}
Jul 07 15:26:56.953392 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:56.953096    2567 scope.go:117] "RemoveContainer" containerID="14f05f2115206b903181d65c6b3054e65b93809624cb98dcdeaf5b5b3498c5a3"
Jul 07 15:26:56.962283 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:56.962259    2567 scope.go:117] "RemoveContainer" containerID="14f05f2115206b903181d65c6b3054e65b93809624cb98dcdeaf5b5b3498c5a3"
Jul 07 15:26:56.962567 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:26:56.962546    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14f05f2115206b903181d65c6b3054e65b93809624cb98dcdeaf5b5b3498c5a3\": container with ID starting with 14f05f2115206b903181d65c6b3054e65b93809624cb98dcdeaf5b5b3498c5a3 not found: ID does not exist" containerID="14f05f2115206b903181d65c6b3054e65b93809624cb98dcdeaf5b5b3498c5a3"
Jul 07 15:26:56.962643 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:56.962580    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14f05f2115206b903181d65c6b3054e65b93809624cb98dcdeaf5b5b3498c5a3"} err="failed to get container status \"14f05f2115206b903181d65c6b3054e65b93809624cb98dcdeaf5b5b3498c5a3\": rpc error: code = NotFound desc = could not find container \"14f05f2115206b903181d65c6b3054e65b93809624cb98dcdeaf5b5b3498c5a3\": container with ID starting with 14f05f2115206b903181d65c6b3054e65b93809624cb98dcdeaf5b5b3498c5a3 not found: ID does not exist"
Jul 07 15:26:56.988634 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:56.988607    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg"]
Jul 07 15:26:56.990470 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:56.990450    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-2d4432df-9964b4b57-mxhvg"]
Jul 07 15:26:58.778259 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:26:58.778224    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a774a0b7-cf1a-483b-a11f-c18d96b762f6" path="/var/lib/kubelet/pods/a774a0b7-cf1a-483b-a11f-c18d96b762f6/volumes"
Jul 07 15:27:02.216565 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.216532    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:27:02.220907 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.220881    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hh557\" (UniqueName: \"kubernetes.io/projected/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-kube-api-access-hh557\") pod \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") "
Jul 07 15:27:02.221051 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.220916    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-credential-socket\") pod \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") "
Jul 07 15:27:02.221051 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.220937    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istio-data\") pod \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") "
Jul 07 15:27:02.221051 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.220965    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-workload-certs\") pod \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") "
Jul 07 15:27:02.221051 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.221046    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istio-envoy\") pod \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") "
Jul 07 15:27:02.221276 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.221094    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-workload-socket\") pod \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") "
Jul 07 15:27:02.221276 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.221140    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istiod-ca-cert\") pod \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") "
Jul 07 15:27:02.221276 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.221168    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istio-token\") pod \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") "
Jul 07 15:27:02.221276 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.221196    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istio-podinfo\") pod \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\" (UID: \"e79b9eaf-ebc3-42c2-98c8-176db1c380d6\") "
Jul 07 15:27:02.221477 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.221253    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e79b9eaf-ebc3-42c2-98c8-176db1c380d6" (UID: "e79b9eaf-ebc3-42c2-98c8-176db1c380d6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:27:02.221477 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.221328    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e79b9eaf-ebc3-42c2-98c8-176db1c380d6" (UID: "e79b9eaf-ebc3-42c2-98c8-176db1c380d6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:27:02.221477 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.221360    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e79b9eaf-ebc3-42c2-98c8-176db1c380d6" (UID: "e79b9eaf-ebc3-42c2-98c8-176db1c380d6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:27:02.221800 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.221525    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-credential-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:27:02.221800 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.221545    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-workload-certs\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:27:02.221800 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.221561    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-workload-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:27:02.221800 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.221603    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e79b9eaf-ebc3-42c2-98c8-176db1c380d6" (UID: "e79b9eaf-ebc3-42c2-98c8-176db1c380d6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:27:02.222046 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.222028    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istio-data" (OuterVolumeSpecName: "istio-data") pod "e79b9eaf-ebc3-42c2-98c8-176db1c380d6" (UID: "e79b9eaf-ebc3-42c2-98c8-176db1c380d6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:27:02.223316 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.223284    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e79b9eaf-ebc3-42c2-98c8-176db1c380d6" (UID: "e79b9eaf-ebc3-42c2-98c8-176db1c380d6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:27:02.223421 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.223359    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-kube-api-access-hh557" (OuterVolumeSpecName: "kube-api-access-hh557") pod "e79b9eaf-ebc3-42c2-98c8-176db1c380d6" (UID: "e79b9eaf-ebc3-42c2-98c8-176db1c380d6"). InnerVolumeSpecName "kube-api-access-hh557". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:27:02.223769 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.223740    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e79b9eaf-ebc3-42c2-98c8-176db1c380d6" (UID: "e79b9eaf-ebc3-42c2-98c8-176db1c380d6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 15:27:02.223769 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.223760    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istio-token" (OuterVolumeSpecName: "istio-token") pod "e79b9eaf-ebc3-42c2-98c8-176db1c380d6" (UID: "e79b9eaf-ebc3-42c2-98c8-176db1c380d6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:27:02.322000 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.321959    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istio-envoy\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:27:02.322000 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.321996    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istiod-ca-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:27:02.322000 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.322006    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istio-token\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:27:02.322260 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.322014    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istio-podinfo\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:27:02.322260 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.322042    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hh557\" (UniqueName: \"kubernetes.io/projected/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-kube-api-access-hh557\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:27:02.322260 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.322053    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e79b9eaf-ebc3-42c2-98c8-176db1c380d6-istio-data\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:27:02.988318 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.988281    2567 generic.go:356] "Generic (PLEG): container finished" podID="e79b9eaf-ebc3-42c2-98c8-176db1c380d6" containerID="75c83be78d740842b6d536edc2e2a79591b9b6fa41d50827236d068a7ffd5178" exitCode=0
Jul 07 15:27:02.988517 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.988342    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"
Jul 07 15:27:02.988517 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.988368    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj" event={"ID":"e79b9eaf-ebc3-42c2-98c8-176db1c380d6","Type":"ContainerDied","Data":"75c83be78d740842b6d536edc2e2a79591b9b6fa41d50827236d068a7ffd5178"}
Jul 07 15:27:02.988517 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.988432    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj" event={"ID":"e79b9eaf-ebc3-42c2-98c8-176db1c380d6","Type":"ContainerDied","Data":"f5fec833d1b7b289c44bd98d1b9a843b09e7a4fbd3fff448de68f3efd752cffc"}
Jul 07 15:27:02.988517 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.988463    2567 scope.go:117] "RemoveContainer" containerID="75c83be78d740842b6d536edc2e2a79591b9b6fa41d50827236d068a7ffd5178"
Jul 07 15:27:02.997703 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.997683    2567 scope.go:117] "RemoveContainer" containerID="75c83be78d740842b6d536edc2e2a79591b9b6fa41d50827236d068a7ffd5178"
Jul 07 15:27:02.998051 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:27:02.998028    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75c83be78d740842b6d536edc2e2a79591b9b6fa41d50827236d068a7ffd5178\": container with ID starting with 75c83be78d740842b6d536edc2e2a79591b9b6fa41d50827236d068a7ffd5178 not found: ID does not exist" containerID="75c83be78d740842b6d536edc2e2a79591b9b6fa41d50827236d068a7ffd5178"
Jul 07 15:27:02.998143 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:02.998059    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75c83be78d740842b6d536edc2e2a79591b9b6fa41d50827236d068a7ffd5178"} err="failed to get container status \"75c83be78d740842b6d536edc2e2a79591b9b6fa41d50827236d068a7ffd5178\": rpc error: code = NotFound desc = could not find container \"75c83be78d740842b6d536edc2e2a79591b9b6fa41d50827236d068a7ffd5178\": container with ID starting with 75c83be78d740842b6d536edc2e2a79591b9b6fa41d50827236d068a7ffd5178 not found: ID does not exist"
Jul 07 15:27:03.014416 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:03.014385    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"]
Jul 07 15:27:03.019419 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:03.019393    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-2d4432df-openshift-default-999c6fbdc-2mhsj"]
Jul 07 15:27:04.778142 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:27:04.778107    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e79b9eaf-ebc3-42c2-98c8-176db1c380d6" path="/var/lib/kubelet/pods/e79b9eaf-ebc3-42c2-98c8-176db1c380d6/volumes"
Jul 07 15:28:43.735502 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.735470    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"]
Jul 07 15:28:43.735977 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.735932    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a774a0b7-cf1a-483b-a11f-c18d96b762f6" containerName="maas-api"
Jul 07 15:28:43.735977 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.735945    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a774a0b7-cf1a-483b-a11f-c18d96b762f6" containerName="maas-api"
Jul 07 15:28:43.735977 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.735963    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e79b9eaf-ebc3-42c2-98c8-176db1c380d6" containerName="istio-proxy"
Jul 07 15:28:43.735977 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.735971    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e79b9eaf-ebc3-42c2-98c8-176db1c380d6" containerName="istio-proxy"
Jul 07 15:28:43.736116 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.736051    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a774a0b7-cf1a-483b-a11f-c18d96b762f6" containerName="maas-api"
Jul 07 15:28:43.736116 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.736064    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e79b9eaf-ebc3-42c2-98c8-176db1c380d6" containerName="istio-proxy"
Jul 07 15:28:43.739147 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.739131    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:43.742378 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.742356    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-5f236d3d-openshift-default-dockercfg-m22g4\""
Jul 07 15:28:43.767457 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.767428    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"]
Jul 07 15:28:43.843655 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.843618    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-workload-certs\") pod \"e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") " pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:43.843874 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.843767    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-credential-socket\") pod \"e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") " pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:43.843874 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.843799    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/115b743c-2447-4282-a3d7-9772f0bd5011-istio-token\") pod \"e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") " pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:43.843874 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.843818    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/115b743c-2447-4282-a3d7-9772f0bd5011-istiod-ca-cert\") pod \"e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") " pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:43.843874 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.843851    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-workload-socket\") pod \"e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") " pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:43.843874 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.843873    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-istio-envoy\") pod \"e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") " pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:43.844232 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.843907    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7xs9\" (UniqueName: \"kubernetes.io/projected/115b743c-2447-4282-a3d7-9772f0bd5011-kube-api-access-p7xs9\") pod \"e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") " pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:43.844232 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.843961    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-istio-data\") pod \"e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") " pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:43.844232 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.844008    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/115b743c-2447-4282-a3d7-9772f0bd5011-istio-podinfo\") pod \"e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") " pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:43.944917 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.944884    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/115b743c-2447-4282-a3d7-9772f0bd5011-istio-podinfo\") pod \"e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") " pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:43.944917 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.944931    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-workload-certs\") pod \"e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") " pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:43.945181 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.945003    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-credential-socket\") pod \"e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") " pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:43.945181 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.945032    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/115b743c-2447-4282-a3d7-9772f0bd5011-istio-token\") pod \"e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") " pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:43.945181 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.945058    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/115b743c-2447-4282-a3d7-9772f0bd5011-istiod-ca-cert\") pod \"e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") " pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:43.945181 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.945088    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-workload-socket\") pod \"e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") " pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:43.945181 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.945111    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-istio-envoy\") pod \"e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") " pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:43.945181 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.945139    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7xs9\" (UniqueName: \"kubernetes.io/projected/115b743c-2447-4282-a3d7-9772f0bd5011-kube-api-access-p7xs9\") pod \"e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") " pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:43.945181 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.945180    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-istio-data\") pod \"e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") " pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:43.945538 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.945455    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-credential-socket\") pod \"e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") " pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:43.945538 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.945518    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-workload-certs\") pod \"e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") " pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:43.945714 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.945692    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-workload-socket\") pod \"e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") " pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:43.945962 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.945927    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/115b743c-2447-4282-a3d7-9772f0bd5011-istiod-ca-cert\") pod \"e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") " pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:43.946072 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.945958    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-istio-data\") pod \"e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") " pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:43.947700 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.947676    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/115b743c-2447-4282-a3d7-9772f0bd5011-istio-podinfo\") pod \"e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") " pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:43.947874 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.947856    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-istio-envoy\") pod \"e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") " pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:43.955190 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.955160    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7xs9\" (UniqueName: \"kubernetes.io/projected/115b743c-2447-4282-a3d7-9772f0bd5011-kube-api-access-p7xs9\") pod \"e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") " pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:43.955280 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:43.955242    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/115b743c-2447-4282-a3d7-9772f0bd5011-istio-token\") pod \"e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") " pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:44.049980 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:44.049944    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:44.201578 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:44.201551    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"]
Jul 07 15:28:44.204568 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:28:44.204528    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod115b743c_2447_4282_a3d7_9772f0bd5011.slice/crio-ba4601ebf2ba201d5fe1642841da6cf05dd33607bcba70ef6fc96f4c82e65c1d WatchSource:0}: Error finding container ba4601ebf2ba201d5fe1642841da6cf05dd33607bcba70ef6fc96f4c82e65c1d: Status 404 returned error can't find the container with id ba4601ebf2ba201d5fe1642841da6cf05dd33607bcba70ef6fc96f4c82e65c1d
Jul 07 15:28:44.207418 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:44.207378    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:28:44.207528 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:44.207474    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:28:44.207528 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:44.207520    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:28:44.523554 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:44.523470    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd" event={"ID":"115b743c-2447-4282-a3d7-9772f0bd5011","Type":"ContainerStarted","Data":"bffa26e9ee9e539a779b2752565f8e51cb983c1fddb6aad970748b50c34665bd"}
Jul 07 15:28:44.523554 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:44.523522    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd" event={"ID":"115b743c-2447-4282-a3d7-9772f0bd5011","Type":"ContainerStarted","Data":"ba4601ebf2ba201d5fe1642841da6cf05dd33607bcba70ef6fc96f4c82e65c1d"}
Jul 07 15:28:44.545929 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:44.545875    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd" podStartSLOduration=1.545858044 podStartE2EDuration="1.545858044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:28:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:28:44.543730455 +0000 UTC m=+1692.340267186" watchObservedRunningTime="2026-07-07 15:28:44.545858044 +0000 UTC m=+1692.342394772"
Jul 07 15:28:45.050951 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:45.050912    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:45.056205 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:45.056181    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:45.529062 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:45.528955    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:45.530027 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:45.530003    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:28:46.680963 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:46.680927    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh"]
Jul 07 15:28:46.684922 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:46.684901    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh"
Jul 07 15:28:46.687970 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:46.687943    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-5f236d3d-serving-cert\""
Jul 07 15:28:46.697385 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:46.697357    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh"]
Jul 07 15:28:46.781938 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:46.781907    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d35a9cf6-bff0-4c53-9040-335dd5a13f38-openshift-service-ca\") pod \"maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh\" (UID: \"d35a9cf6-bff0-4c53-9040-335dd5a13f38\") " pod="opendatahub/maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh"
Jul 07 15:28:46.782109 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:46.781957    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bfwf\" (UniqueName: \"kubernetes.io/projected/d35a9cf6-bff0-4c53-9040-335dd5a13f38-kube-api-access-9bfwf\") pod \"maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh\" (UID: \"d35a9cf6-bff0-4c53-9040-335dd5a13f38\") " pod="opendatahub/maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh"
Jul 07 15:28:46.782109 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:46.782025    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d35a9cf6-bff0-4c53-9040-335dd5a13f38-maas-api-tls\") pod \"maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh\" (UID: \"d35a9cf6-bff0-4c53-9040-335dd5a13f38\") " pod="opendatahub/maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh"
Jul 07 15:28:46.882609 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:46.882563    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d35a9cf6-bff0-4c53-9040-335dd5a13f38-openshift-service-ca\") pod \"maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh\" (UID: \"d35a9cf6-bff0-4c53-9040-335dd5a13f38\") " pod="opendatahub/maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh"
Jul 07 15:28:46.882816 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:46.882626    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bfwf\" (UniqueName: \"kubernetes.io/projected/d35a9cf6-bff0-4c53-9040-335dd5a13f38-kube-api-access-9bfwf\") pod \"maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh\" (UID: \"d35a9cf6-bff0-4c53-9040-335dd5a13f38\") " pod="opendatahub/maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh"
Jul 07 15:28:46.882816 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:46.882661    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d35a9cf6-bff0-4c53-9040-335dd5a13f38-maas-api-tls\") pod \"maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh\" (UID: \"d35a9cf6-bff0-4c53-9040-335dd5a13f38\") " pod="opendatahub/maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh"
Jul 07 15:28:46.882999 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:28:46.882876    2567 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-5f236d3d-serving-cert: secret "maas-api-e2e-mt-5f236d3d-serving-cert" not found
Jul 07 15:28:46.882999 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:28:46.882967    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d35a9cf6-bff0-4c53-9040-335dd5a13f38-maas-api-tls podName:d35a9cf6-bff0-4c53-9040-335dd5a13f38 nodeName:}" failed. No retries permitted until 2026-07-07 15:28:47.382943296 +0000 UTC m=+1695.179480004 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d35a9cf6-bff0-4c53-9040-335dd5a13f38-maas-api-tls") pod "maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh" (UID: "d35a9cf6-bff0-4c53-9040-335dd5a13f38") : secret "maas-api-e2e-mt-5f236d3d-serving-cert" not found
Jul 07 15:28:46.883386 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:46.883366    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d35a9cf6-bff0-4c53-9040-335dd5a13f38-openshift-service-ca\") pod \"maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh\" (UID: \"d35a9cf6-bff0-4c53-9040-335dd5a13f38\") " pod="opendatahub/maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh"
Jul 07 15:28:46.897937 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:46.897909    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bfwf\" (UniqueName: \"kubernetes.io/projected/d35a9cf6-bff0-4c53-9040-335dd5a13f38-kube-api-access-9bfwf\") pod \"maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh\" (UID: \"d35a9cf6-bff0-4c53-9040-335dd5a13f38\") " pod="opendatahub/maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh"
Jul 07 15:28:47.387516 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:47.387481    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d35a9cf6-bff0-4c53-9040-335dd5a13f38-maas-api-tls\") pod \"maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh\" (UID: \"d35a9cf6-bff0-4c53-9040-335dd5a13f38\") " pod="opendatahub/maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh"
Jul 07 15:28:47.389951 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:47.389919    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d35a9cf6-bff0-4c53-9040-335dd5a13f38-maas-api-tls\") pod \"maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh\" (UID: \"d35a9cf6-bff0-4c53-9040-335dd5a13f38\") " pod="opendatahub/maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh"
Jul 07 15:28:47.598295 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:47.598264    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh"
Jul 07 15:28:47.754031 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:47.753993    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh"]
Jul 07 15:28:47.756447 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:28:47.756408    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd35a9cf6_bff0_4c53_9040_335dd5a13f38.slice/crio-9f76f44143d1c8fbcb8456c6d9864099b13af8bd86689138dfdc9cd6dc868fb8 WatchSource:0}: Error finding container 9f76f44143d1c8fbcb8456c6d9864099b13af8bd86689138dfdc9cd6dc868fb8: Status 404 returned error can't find the container with id 9f76f44143d1c8fbcb8456c6d9864099b13af8bd86689138dfdc9cd6dc868fb8
Jul 07 15:28:48.546463 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:48.546429    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh" event={"ID":"d35a9cf6-bff0-4c53-9040-335dd5a13f38","Type":"ContainerStarted","Data":"23cbe9dc66be3692554678a7b9c1a05d502404e5ce0729c31054404df3f9b112"}
Jul 07 15:28:48.546662 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:48.546490    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh" event={"ID":"d35a9cf6-bff0-4c53-9040-335dd5a13f38","Type":"ContainerStarted","Data":"9f76f44143d1c8fbcb8456c6d9864099b13af8bd86689138dfdc9cd6dc868fb8"}
Jul 07 15:28:48.546662 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:48.546517    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh"
Jul 07 15:28:48.573212 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:48.573158    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh" podStartSLOduration=1.9708612269999999 podStartE2EDuration="2.573142299s" totalImagesPullingTime="602.281072ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:28:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:28:48.572807132 +0000 UTC m=+1696.369343860" watchObservedRunningTime="2026-07-07 15:28:48.573142299 +0000 UTC m=+1696.369679027"
Jul 07 15:28:54.556572 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:54.556540    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh"
Jul 07 15:28:59.901920 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:59.901886    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh"]
Jul 07 15:28:59.902354 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:28:59.902237    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh" podUID="d35a9cf6-bff0-4c53-9040-335dd5a13f38" containerName="maas-api" containerID="cri-o://23cbe9dc66be3692554678a7b9c1a05d502404e5ce0729c31054404df3f9b112" gracePeriod=30
Jul 07 15:29:00.167483 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:00.167453    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh"
Jul 07 15:29:00.311628 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:00.311578    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d35a9cf6-bff0-4c53-9040-335dd5a13f38-maas-api-tls\") pod \"d35a9cf6-bff0-4c53-9040-335dd5a13f38\" (UID: \"d35a9cf6-bff0-4c53-9040-335dd5a13f38\") "
Jul 07 15:29:00.311817 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:00.311671    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9bfwf\" (UniqueName: \"kubernetes.io/projected/d35a9cf6-bff0-4c53-9040-335dd5a13f38-kube-api-access-9bfwf\") pod \"d35a9cf6-bff0-4c53-9040-335dd5a13f38\" (UID: \"d35a9cf6-bff0-4c53-9040-335dd5a13f38\") "
Jul 07 15:29:00.311817 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:00.311742    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d35a9cf6-bff0-4c53-9040-335dd5a13f38-openshift-service-ca\") pod \"d35a9cf6-bff0-4c53-9040-335dd5a13f38\" (UID: \"d35a9cf6-bff0-4c53-9040-335dd5a13f38\") "
Jul 07 15:29:00.312177 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:00.312152    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d35a9cf6-bff0-4c53-9040-335dd5a13f38-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d35a9cf6-bff0-4c53-9040-335dd5a13f38" (UID: "d35a9cf6-bff0-4c53-9040-335dd5a13f38"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:29:00.314027 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:00.313995    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d35a9cf6-bff0-4c53-9040-335dd5a13f38-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d35a9cf6-bff0-4c53-9040-335dd5a13f38" (UID: "d35a9cf6-bff0-4c53-9040-335dd5a13f38"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:29:00.314137 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:00.314095    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d35a9cf6-bff0-4c53-9040-335dd5a13f38-kube-api-access-9bfwf" (OuterVolumeSpecName: "kube-api-access-9bfwf") pod "d35a9cf6-bff0-4c53-9040-335dd5a13f38" (UID: "d35a9cf6-bff0-4c53-9040-335dd5a13f38"). InnerVolumeSpecName "kube-api-access-9bfwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:29:00.413318 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:00.413217    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d35a9cf6-bff0-4c53-9040-335dd5a13f38-maas-api-tls\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:29:00.413318 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:00.413262    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bfwf\" (UniqueName: \"kubernetes.io/projected/d35a9cf6-bff0-4c53-9040-335dd5a13f38-kube-api-access-9bfwf\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:29:00.413318 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:00.413275    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d35a9cf6-bff0-4c53-9040-335dd5a13f38-openshift-service-ca\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:29:00.530098 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:00.530065    2567 patch_prober.go:28] interesting pod/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Jul 07 15:29:00.530255 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:00.530122    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd" podUID="115b743c-2447-4282-a3d7-9772f0bd5011" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Jul 07 15:29:00.608789 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:00.608751    2567 generic.go:356] "Generic (PLEG): container finished" podID="115b743c-2447-4282-a3d7-9772f0bd5011" containerID="bffa26e9ee9e539a779b2752565f8e51cb983c1fddb6aad970748b50c34665bd" exitCode=0
Jul 07 15:29:00.608979 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:00.608819    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd" event={"ID":"115b743c-2447-4282-a3d7-9772f0bd5011","Type":"ContainerDied","Data":"bffa26e9ee9e539a779b2752565f8e51cb983c1fddb6aad970748b50c34665bd"}
Jul 07 15:29:00.609249 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:00.609222    2567 scope.go:117] "RemoveContainer" containerID="bffa26e9ee9e539a779b2752565f8e51cb983c1fddb6aad970748b50c34665bd"
Jul 07 15:29:00.610352 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:00.610329    2567 generic.go:356] "Generic (PLEG): container finished" podID="d35a9cf6-bff0-4c53-9040-335dd5a13f38" containerID="23cbe9dc66be3692554678a7b9c1a05d502404e5ce0729c31054404df3f9b112" exitCode=0
Jul 07 15:29:00.610479 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:00.610350    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:29:00.610479 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:00.610373    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh"
Jul 07 15:29:00.610479 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:00.610376    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh" event={"ID":"d35a9cf6-bff0-4c53-9040-335dd5a13f38","Type":"ContainerDied","Data":"23cbe9dc66be3692554678a7b9c1a05d502404e5ce0729c31054404df3f9b112"}
Jul 07 15:29:00.610479 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:00.610442    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh" event={"ID":"d35a9cf6-bff0-4c53-9040-335dd5a13f38","Type":"ContainerDied","Data":"9f76f44143d1c8fbcb8456c6d9864099b13af8bd86689138dfdc9cd6dc868fb8"}
Jul 07 15:29:00.610479 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:00.610446    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:29:00.610479 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:00.610474    2567 scope.go:117] "RemoveContainer" containerID="23cbe9dc66be3692554678a7b9c1a05d502404e5ce0729c31054404df3f9b112"
Jul 07 15:29:00.610793 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:00.610489    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:29:00.623529 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:00.623504    2567 scope.go:117] "RemoveContainer" containerID="23cbe9dc66be3692554678a7b9c1a05d502404e5ce0729c31054404df3f9b112"
Jul 07 15:29:00.623857 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:29:00.623808    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23cbe9dc66be3692554678a7b9c1a05d502404e5ce0729c31054404df3f9b112\": container with ID starting with 23cbe9dc66be3692554678a7b9c1a05d502404e5ce0729c31054404df3f9b112 not found: ID does not exist" containerID="23cbe9dc66be3692554678a7b9c1a05d502404e5ce0729c31054404df3f9b112"
Jul 07 15:29:00.623988 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:00.623868    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23cbe9dc66be3692554678a7b9c1a05d502404e5ce0729c31054404df3f9b112"} err="failed to get container status \"23cbe9dc66be3692554678a7b9c1a05d502404e5ce0729c31054404df3f9b112\": rpc error: code = NotFound desc = could not find container \"23cbe9dc66be3692554678a7b9c1a05d502404e5ce0729c31054404df3f9b112\": container with ID starting with 23cbe9dc66be3692554678a7b9c1a05d502404e5ce0729c31054404df3f9b112 not found: ID does not exist"
Jul 07 15:29:00.670195 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:00.670167    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh"]
Jul 07 15:29:00.673322 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:00.673296    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-5f236d3d-9dbc98fff-w7zgh"]
Jul 07 15:29:00.779604 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:00.779571    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d35a9cf6-bff0-4c53-9040-335dd5a13f38" path="/var/lib/kubelet/pods/d35a9cf6-bff0-4c53-9040-335dd5a13f38/volumes"
Jul 07 15:29:01.622314 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:01.622279    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd" event={"ID":"115b743c-2447-4282-a3d7-9772f0bd5011","Type":"ContainerStarted","Data":"26e0fa82542d62828fc3ec91343502c219badbdc90b77e31ec580a82396a2b30"}
Jul 07 15:29:02.050530 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:02.050499    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:29:02.050861 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:02.050810    2567 patch_prober.go:28] interesting pod/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Jul 07 15:29:02.050919 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:02.050898    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd" podUID="115b743c-2447-4282-a3d7-9772f0bd5011" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Jul 07 15:29:03.051175 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:03.051134    2567 patch_prober.go:28] interesting pod/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Jul 07 15:29:03.051697 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:03.051200    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd" podUID="115b743c-2447-4282-a3d7-9772f0bd5011" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Jul 07 15:29:04.050668 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:04.050626    2567 patch_prober.go:28] interesting pod/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Jul 07 15:29:04.050918 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:04.050692    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd" podUID="115b743c-2447-4282-a3d7-9772f0bd5011" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Jul 07 15:29:05.050470 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:05.050431    2567 patch_prober.go:28] interesting pod/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Jul 07 15:29:05.050916 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:05.050498    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd" podUID="115b743c-2447-4282-a3d7-9772f0bd5011" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Jul 07 15:29:06.055293 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:06.055253    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:29:06.055774 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:06.055644    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:29:06.056296 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:06.056282    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:29:06.764640 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:06.764609    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"]
Jul 07 15:29:07.661307 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:07.661271    2567 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd" secret="" err="secret \"e2e-mt-5f236d3d-openshift-default-dockercfg-m22g4\" not found"
Jul 07 15:29:08.666130 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:08.666091    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd" podUID="115b743c-2447-4282-a3d7-9772f0bd5011" containerName="istio-proxy" containerID="cri-o://26e0fa82542d62828fc3ec91343502c219badbdc90b77e31ec580a82396a2b30" gracePeriod=30
Jul 07 15:29:13.937686 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:13.937623    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:29:14.050447 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.050397    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-istio-data\") pod \"115b743c-2447-4282-a3d7-9772f0bd5011\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") "
Jul 07 15:29:14.050673 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.050466    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/115b743c-2447-4282-a3d7-9772f0bd5011-istiod-ca-cert\") pod \"115b743c-2447-4282-a3d7-9772f0bd5011\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") "
Jul 07 15:29:14.050673 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.050502    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7xs9\" (UniqueName: \"kubernetes.io/projected/115b743c-2447-4282-a3d7-9772f0bd5011-kube-api-access-p7xs9\") pod \"115b743c-2447-4282-a3d7-9772f0bd5011\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") "
Jul 07 15:29:14.050673 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.050576    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/115b743c-2447-4282-a3d7-9772f0bd5011-istio-token\") pod \"115b743c-2447-4282-a3d7-9772f0bd5011\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") "
Jul 07 15:29:14.050673 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.050617    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-workload-socket\") pod \"115b743c-2447-4282-a3d7-9772f0bd5011\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") "
Jul 07 15:29:14.050673 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.050653    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-credential-socket\") pod \"115b743c-2447-4282-a3d7-9772f0bd5011\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") "
Jul 07 15:29:14.050996 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.050692    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/115b743c-2447-4282-a3d7-9772f0bd5011-istio-podinfo\") pod \"115b743c-2447-4282-a3d7-9772f0bd5011\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") "
Jul 07 15:29:14.050996 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.050729    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-istio-envoy\") pod \"115b743c-2447-4282-a3d7-9772f0bd5011\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") "
Jul 07 15:29:14.050996 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.050777    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-workload-certs\") pod \"115b743c-2447-4282-a3d7-9772f0bd5011\" (UID: \"115b743c-2447-4282-a3d7-9772f0bd5011\") "
Jul 07 15:29:14.050996 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.050955    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "115b743c-2447-4282-a3d7-9772f0bd5011" (UID: "115b743c-2447-4282-a3d7-9772f0bd5011"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:29:14.050996 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.050971    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "115b743c-2447-4282-a3d7-9772f0bd5011" (UID: "115b743c-2447-4282-a3d7-9772f0bd5011"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:29:14.050996 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.050954    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/115b743c-2447-4282-a3d7-9772f0bd5011-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "115b743c-2447-4282-a3d7-9772f0bd5011" (UID: "115b743c-2447-4282-a3d7-9772f0bd5011"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:29:14.051306 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.051165    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "115b743c-2447-4282-a3d7-9772f0bd5011" (UID: "115b743c-2447-4282-a3d7-9772f0bd5011"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:29:14.051306 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.051207    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/115b743c-2447-4282-a3d7-9772f0bd5011-istiod-ca-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:29:14.051306 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.051230    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-workload-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:29:14.051306 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.051243    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-credential-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:29:14.051490 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.051467    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-istio-data" (OuterVolumeSpecName: "istio-data") pod "115b743c-2447-4282-a3d7-9772f0bd5011" (UID: "115b743c-2447-4282-a3d7-9772f0bd5011"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:29:14.052899 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.052874    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/115b743c-2447-4282-a3d7-9772f0bd5011-istio-token" (OuterVolumeSpecName: "istio-token") pod "115b743c-2447-4282-a3d7-9772f0bd5011" (UID: "115b743c-2447-4282-a3d7-9772f0bd5011"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:29:14.053178 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.052984    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/115b743c-2447-4282-a3d7-9772f0bd5011-kube-api-access-p7xs9" (OuterVolumeSpecName: "kube-api-access-p7xs9") pod "115b743c-2447-4282-a3d7-9772f0bd5011" (UID: "115b743c-2447-4282-a3d7-9772f0bd5011"). InnerVolumeSpecName "kube-api-access-p7xs9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:29:14.053178 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.053081    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "115b743c-2447-4282-a3d7-9772f0bd5011" (UID: "115b743c-2447-4282-a3d7-9772f0bd5011"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:29:14.053270 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.053231    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/115b743c-2447-4282-a3d7-9772f0bd5011-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "115b743c-2447-4282-a3d7-9772f0bd5011" (UID: "115b743c-2447-4282-a3d7-9772f0bd5011"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 15:29:14.152657 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.152617    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7xs9\" (UniqueName: \"kubernetes.io/projected/115b743c-2447-4282-a3d7-9772f0bd5011-kube-api-access-p7xs9\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:29:14.152657 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.152651    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/115b743c-2447-4282-a3d7-9772f0bd5011-istio-token\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:29:14.152657 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.152664    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/115b743c-2447-4282-a3d7-9772f0bd5011-istio-podinfo\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:29:14.152937 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.152676    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-istio-envoy\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:29:14.152937 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.152689    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-workload-certs\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:29:14.152937 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.152699    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/115b743c-2447-4282-a3d7-9772f0bd5011-istio-data\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:29:14.697109 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.697069    2567 generic.go:356] "Generic (PLEG): container finished" podID="115b743c-2447-4282-a3d7-9772f0bd5011" containerID="26e0fa82542d62828fc3ec91343502c219badbdc90b77e31ec580a82396a2b30" exitCode=0
Jul 07 15:29:14.697296 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.697125    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"
Jul 07 15:29:14.697296 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.697153    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd" event={"ID":"115b743c-2447-4282-a3d7-9772f0bd5011","Type":"ContainerDied","Data":"26e0fa82542d62828fc3ec91343502c219badbdc90b77e31ec580a82396a2b30"}
Jul 07 15:29:14.697296 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.697223    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd" event={"ID":"115b743c-2447-4282-a3d7-9772f0bd5011","Type":"ContainerDied","Data":"ba4601ebf2ba201d5fe1642841da6cf05dd33607bcba70ef6fc96f4c82e65c1d"}
Jul 07 15:29:14.697296 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.697254    2567 scope.go:117] "RemoveContainer" containerID="26e0fa82542d62828fc3ec91343502c219badbdc90b77e31ec580a82396a2b30"
Jul 07 15:29:14.707913 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.707894    2567 scope.go:117] "RemoveContainer" containerID="bffa26e9ee9e539a779b2752565f8e51cb983c1fddb6aad970748b50c34665bd"
Jul 07 15:29:14.716934 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.716819    2567 scope.go:117] "RemoveContainer" containerID="26e0fa82542d62828fc3ec91343502c219badbdc90b77e31ec580a82396a2b30"
Jul 07 15:29:14.717248 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:29:14.717224    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26e0fa82542d62828fc3ec91343502c219badbdc90b77e31ec580a82396a2b30\": container with ID starting with 26e0fa82542d62828fc3ec91343502c219badbdc90b77e31ec580a82396a2b30 not found: ID does not exist" containerID="26e0fa82542d62828fc3ec91343502c219badbdc90b77e31ec580a82396a2b30"
Jul 07 15:29:14.717306 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.717259    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26e0fa82542d62828fc3ec91343502c219badbdc90b77e31ec580a82396a2b30"} err="failed to get container status \"26e0fa82542d62828fc3ec91343502c219badbdc90b77e31ec580a82396a2b30\": rpc error: code = NotFound desc = could not find container \"26e0fa82542d62828fc3ec91343502c219badbdc90b77e31ec580a82396a2b30\": container with ID starting with 26e0fa82542d62828fc3ec91343502c219badbdc90b77e31ec580a82396a2b30 not found: ID does not exist"
Jul 07 15:29:14.717306 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.717278    2567 scope.go:117] "RemoveContainer" containerID="bffa26e9ee9e539a779b2752565f8e51cb983c1fddb6aad970748b50c34665bd"
Jul 07 15:29:14.717517 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:29:14.717500    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bffa26e9ee9e539a779b2752565f8e51cb983c1fddb6aad970748b50c34665bd\": container with ID starting with bffa26e9ee9e539a779b2752565f8e51cb983c1fddb6aad970748b50c34665bd not found: ID does not exist" containerID="bffa26e9ee9e539a779b2752565f8e51cb983c1fddb6aad970748b50c34665bd"
Jul 07 15:29:14.717585 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.717521    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bffa26e9ee9e539a779b2752565f8e51cb983c1fddb6aad970748b50c34665bd"} err="failed to get container status \"bffa26e9ee9e539a779b2752565f8e51cb983c1fddb6aad970748b50c34665bd\": rpc error: code = NotFound desc = could not find container \"bffa26e9ee9e539a779b2752565f8e51cb983c1fddb6aad970748b50c34665bd\": container with ID starting with bffa26e9ee9e539a779b2752565f8e51cb983c1fddb6aad970748b50c34665bd not found: ID does not exist"
Jul 07 15:29:14.723575 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.723552    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"]
Jul 07 15:29:14.727751 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.727722    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-5f236d3d-openshift-default-5874ddc5c9-vblgd"]
Jul 07 15:29:14.777788 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:14.777755    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="115b743c-2447-4282-a3d7-9772f0bd5011" path="/var/lib/kubelet/pods/115b743c-2447-4282-a3d7-9772f0bd5011/volumes"
Jul 07 15:29:28.242611 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.242576    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"]
Jul 07 15:29:28.243231 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.243215    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="115b743c-2447-4282-a3d7-9772f0bd5011" containerName="istio-proxy"
Jul 07 15:29:28.243287 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.243235    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="115b743c-2447-4282-a3d7-9772f0bd5011" containerName="istio-proxy"
Jul 07 15:29:28.243287 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.243254    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="115b743c-2447-4282-a3d7-9772f0bd5011" containerName="istio-proxy"
Jul 07 15:29:28.243287 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.243263    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="115b743c-2447-4282-a3d7-9772f0bd5011" containerName="istio-proxy"
Jul 07 15:29:28.243287 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.243278    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d35a9cf6-bff0-4c53-9040-335dd5a13f38" containerName="maas-api"
Jul 07 15:29:28.243417 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.243287    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d35a9cf6-bff0-4c53-9040-335dd5a13f38" containerName="maas-api"
Jul 07 15:29:28.243417 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.243376    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="115b743c-2447-4282-a3d7-9772f0bd5011" containerName="istio-proxy"
Jul 07 15:29:28.243417 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.243388    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="115b743c-2447-4282-a3d7-9772f0bd5011" containerName="istio-proxy"
Jul 07 15:29:28.243417 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.243401    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d35a9cf6-bff0-4c53-9040-335dd5a13f38" containerName="maas-api"
Jul 07 15:29:28.246716 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.246699    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.250662 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.250641    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-c55c8ec1-openshift-default-dockercfg-cd2xb\""
Jul 07 15:29:28.262024 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.261994    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"]
Jul 07 15:29:28.279059 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.279028    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/213670e0-be84-4c80-86e6-ab501e3b9d40-istio-podinfo\") pod \"e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") " pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.279199 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.279131    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-istio-envoy\") pod \"e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") " pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.279199 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.279175    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-credential-socket\") pod \"e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") " pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.279319 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.279208    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-istio-data\") pod \"e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") " pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.279319 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.279231    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/213670e0-be84-4c80-86e6-ab501e3b9d40-istio-token\") pod \"e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") " pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.279319 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.279285    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww64v\" (UniqueName: \"kubernetes.io/projected/213670e0-be84-4c80-86e6-ab501e3b9d40-kube-api-access-ww64v\") pod \"e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") " pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.279478 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.279389    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-workload-certs\") pod \"e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") " pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.279478 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.279426    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-workload-socket\") pod \"e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") " pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.279478 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.279450    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/213670e0-be84-4c80-86e6-ab501e3b9d40-istiod-ca-cert\") pod \"e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") " pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.380125 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.380082    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-workload-certs\") pod \"e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") " pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.380125 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.380128    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-workload-socket\") pod \"e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") " pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.380409 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.380144    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/213670e0-be84-4c80-86e6-ab501e3b9d40-istiod-ca-cert\") pod \"e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") " pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.380409 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.380178    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/213670e0-be84-4c80-86e6-ab501e3b9d40-istio-podinfo\") pod \"e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") " pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.380409 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.380234    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-istio-envoy\") pod \"e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") " pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.380409 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.380267    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-credential-socket\") pod \"e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") " pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.380409 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.380297    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-istio-data\") pod \"e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") " pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.380409 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.380325    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/213670e0-be84-4c80-86e6-ab501e3b9d40-istio-token\") pod \"e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") " pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.380409 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.380348    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww64v\" (UniqueName: \"kubernetes.io/projected/213670e0-be84-4c80-86e6-ab501e3b9d40-kube-api-access-ww64v\") pod \"e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") " pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.380778 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.380578    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-credential-socket\") pod \"e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") " pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.380778 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.380658    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-istio-data\") pod \"e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") " pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.380951 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.380930    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-workload-socket\") pod \"e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") " pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.381281 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.381258    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-workload-certs\") pod \"e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") " pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.381355 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.381290    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/213670e0-be84-4c80-86e6-ab501e3b9d40-istiod-ca-cert\") pod \"e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") " pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.382663 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.382642    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-istio-envoy\") pod \"e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") " pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.383396 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.383377    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/213670e0-be84-4c80-86e6-ab501e3b9d40-istio-podinfo\") pod \"e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") " pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.399800 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.399778    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/213670e0-be84-4c80-86e6-ab501e3b9d40-istio-token\") pod \"e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") " pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.400190 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.400171    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww64v\" (UniqueName: \"kubernetes.io/projected/213670e0-be84-4c80-86e6-ab501e3b9d40-kube-api-access-ww64v\") pod \"e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") " pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.557400 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.557358    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:28.728370 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.728344    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"]
Jul 07 15:29:28.730970 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:29:28.730941    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod213670e0_be84_4c80_86e6_ab501e3b9d40.slice/crio-7028c39bf401b61b77d449c0052040efeb7a9e376eb7aff4a8a8764484eb1456 WatchSource:0}: Error finding container 7028c39bf401b61b77d449c0052040efeb7a9e376eb7aff4a8a8764484eb1456: Status 404 returned error can't find the container with id 7028c39bf401b61b77d449c0052040efeb7a9e376eb7aff4a8a8764484eb1456
Jul 07 15:29:28.733120 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.733085    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:29:28.733220 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.733153    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:29:28.733220 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.733181    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:29:28.770680 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:28.770644    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk" event={"ID":"213670e0-be84-4c80-86e6-ab501e3b9d40","Type":"ContainerStarted","Data":"7028c39bf401b61b77d449c0052040efeb7a9e376eb7aff4a8a8764484eb1456"}
Jul 07 15:29:29.778734 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:29.778699    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk" event={"ID":"213670e0-be84-4c80-86e6-ab501e3b9d40","Type":"ContainerStarted","Data":"5c287ead6a38f4c0a39ac08e23e87d2779798ef591de7540c08d70ff414aade7"}
Jul 07 15:29:29.809803 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:29.809739    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk" podStartSLOduration=1.809718713 podStartE2EDuration="1.809718713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:29:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:29:29.807146011 +0000 UTC m=+1737.603682752" watchObservedRunningTime="2026-07-07 15:29:29.809718713 +0000 UTC m=+1737.606255441"
Jul 07 15:29:30.557720 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:30.557681    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:30.562817 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:30.562791    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:30.783862 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:30.783816    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:30.784858 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:30.784821    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:31.897416 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:31.897385    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph"]
Jul 07 15:29:31.901524 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:31.901500    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph"
Jul 07 15:29:31.904781 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:31.904758    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-c55c8ec1-serving-cert\""
Jul 07 15:29:31.919867 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:31.919801    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph"]
Jul 07 15:29:32.016268 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:32.016226    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4385c52-9a3c-4e29-9a81-9406c8fa54fe-openshift-service-ca\") pod \"maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph\" (UID: \"a4385c52-9a3c-4e29-9a81-9406c8fa54fe\") " pod="opendatahub/maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph"
Jul 07 15:29:32.016452 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:32.016275    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjzsw\" (UniqueName: \"kubernetes.io/projected/a4385c52-9a3c-4e29-9a81-9406c8fa54fe-kube-api-access-bjzsw\") pod \"maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph\" (UID: \"a4385c52-9a3c-4e29-9a81-9406c8fa54fe\") " pod="opendatahub/maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph"
Jul 07 15:29:32.016452 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:32.016347    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4385c52-9a3c-4e29-9a81-9406c8fa54fe-maas-api-tls\") pod \"maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph\" (UID: \"a4385c52-9a3c-4e29-9a81-9406c8fa54fe\") " pod="opendatahub/maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph"
Jul 07 15:29:32.117759 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:32.117719    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4385c52-9a3c-4e29-9a81-9406c8fa54fe-openshift-service-ca\") pod \"maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph\" (UID: \"a4385c52-9a3c-4e29-9a81-9406c8fa54fe\") " pod="opendatahub/maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph"
Jul 07 15:29:32.117759 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:32.117758    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjzsw\" (UniqueName: \"kubernetes.io/projected/a4385c52-9a3c-4e29-9a81-9406c8fa54fe-kube-api-access-bjzsw\") pod \"maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph\" (UID: \"a4385c52-9a3c-4e29-9a81-9406c8fa54fe\") " pod="opendatahub/maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph"
Jul 07 15:29:32.118068 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:32.117918    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4385c52-9a3c-4e29-9a81-9406c8fa54fe-maas-api-tls\") pod \"maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph\" (UID: \"a4385c52-9a3c-4e29-9a81-9406c8fa54fe\") " pod="opendatahub/maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph"
Jul 07 15:29:32.118645 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:32.118570    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4385c52-9a3c-4e29-9a81-9406c8fa54fe-openshift-service-ca\") pod \"maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph\" (UID: \"a4385c52-9a3c-4e29-9a81-9406c8fa54fe\") " pod="opendatahub/maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph"
Jul 07 15:29:32.120484 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:32.120454    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4385c52-9a3c-4e29-9a81-9406c8fa54fe-maas-api-tls\") pod \"maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph\" (UID: \"a4385c52-9a3c-4e29-9a81-9406c8fa54fe\") " pod="opendatahub/maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph"
Jul 07 15:29:32.127249 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:32.127221    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjzsw\" (UniqueName: \"kubernetes.io/projected/a4385c52-9a3c-4e29-9a81-9406c8fa54fe-kube-api-access-bjzsw\") pod \"maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph\" (UID: \"a4385c52-9a3c-4e29-9a81-9406c8fa54fe\") " pod="opendatahub/maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph"
Jul 07 15:29:32.212736 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:32.212643    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph"
Jul 07 15:29:32.362789 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:32.362761    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph"]
Jul 07 15:29:32.364520 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:29:32.364486    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4385c52_9a3c_4e29_9a81_9406c8fa54fe.slice/crio-647af8e511982fff81f9019bdb92b20af33fe3343600997e3ba402278462c84b WatchSource:0}: Error finding container 647af8e511982fff81f9019bdb92b20af33fe3343600997e3ba402278462c84b: Status 404 returned error can't find the container with id 647af8e511982fff81f9019bdb92b20af33fe3343600997e3ba402278462c84b
Jul 07 15:29:32.795250 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:32.795210    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph" event={"ID":"a4385c52-9a3c-4e29-9a81-9406c8fa54fe","Type":"ContainerStarted","Data":"647af8e511982fff81f9019bdb92b20af33fe3343600997e3ba402278462c84b"}
Jul 07 15:29:33.802561 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:33.802523    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph" event={"ID":"a4385c52-9a3c-4e29-9a81-9406c8fa54fe","Type":"ContainerStarted","Data":"ec32c4f3dc301e175b5bf0d8ee67516061727fe2c9557a5febdab3954c35bb02"}
Jul 07 15:29:33.802991 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:33.802674    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph"
Jul 07 15:29:33.839649 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:33.839598    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph" podStartSLOduration=2.348568791 podStartE2EDuration="2.839581764s" totalImagesPullingTime="491.012973ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:29:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:29:33.839317014 +0000 UTC m=+1741.635853742" watchObservedRunningTime="2026-07-07 15:29:33.839581764 +0000 UTC m=+1741.636118491"
Jul 07 15:29:39.814129 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:39.814090    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph"
Jul 07 15:29:46.035340 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:46.034817    2567 patch_prober.go:28] interesting pod/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": read tcp 10.132.0.2:45536->10.132.0.67:15021: read: connection reset by peer" start-of-body=
Jul 07 15:29:46.035340 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:46.034915    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk" podUID="213670e0-be84-4c80-86e6-ab501e3b9d40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": read tcp 10.132.0.2:45536->10.132.0.67:15021: read: connection reset by peer"
Jul 07 15:29:46.881847 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:46.881804    2567 generic.go:356] "Generic (PLEG): container finished" podID="213670e0-be84-4c80-86e6-ab501e3b9d40" containerID="5c287ead6a38f4c0a39ac08e23e87d2779798ef591de7540c08d70ff414aade7" exitCode=0
Jul 07 15:29:46.882035 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:46.881865    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk" event={"ID":"213670e0-be84-4c80-86e6-ab501e3b9d40","Type":"ContainerDied","Data":"5c287ead6a38f4c0a39ac08e23e87d2779798ef591de7540c08d70ff414aade7"}
Jul 07 15:29:46.882267 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:46.882254    2567 scope.go:117] "RemoveContainer" containerID="5c287ead6a38f4c0a39ac08e23e87d2779798ef591de7540c08d70ff414aade7"
Jul 07 15:29:46.883281 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:46.883256    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:29:46.883340 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:46.883317    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:29:46.883376 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:46.883365    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:29:47.889360 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:47.889327    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk" event={"ID":"213670e0-be84-4c80-86e6-ab501e3b9d40","Type":"ContainerStarted","Data":"28fa8d8b1c1682fd1fce031278ad2b235d3d3e4964efedc8b1a73f94ec1b82ee"}
Jul 07 15:29:48.557884 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:48.557805    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:48.558130 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:48.558108    2567 patch_prober.go:28] interesting pod/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Jul 07 15:29:48.558183 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:48.558164    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk" podUID="213670e0-be84-4c80-86e6-ab501e3b9d40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Jul 07 15:29:49.557814 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:49.557775    2567 patch_prober.go:28] interesting pod/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Jul 07 15:29:49.558228 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:49.557884    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk" podUID="213670e0-be84-4c80-86e6-ab501e3b9d40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Jul 07 15:29:50.558636 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:50.558597    2567 patch_prober.go:28] interesting pod/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Jul 07 15:29:50.559057 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:50.558675    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk" podUID="213670e0-be84-4c80-86e6-ab501e3b9d40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Jul 07 15:29:51.558642 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:51.558604    2567 patch_prober.go:28] interesting pod/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Jul 07 15:29:51.559070 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:51.558678    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk" podUID="213670e0-be84-4c80-86e6-ab501e3b9d40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Jul 07 15:29:52.561924 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:52.561893    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:52.562310 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:52.562156    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:29:52.562927 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:29:52.562911    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:30:00.152235 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:00.152191    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-mt-c55c8ec1-29723970-l6rhc"]
Jul 07 15:30:00.156342 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:00.156322    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-mt-c55c8ec1-29723970-l6rhc"
Jul 07 15:30:00.167739 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:00.167705    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-mt-c55c8ec1-29723970-l6rhc"]
Jul 07 15:30:00.299703 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:00.299660    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djfj5\" (UniqueName: \"kubernetes.io/projected/d6c1a612-14bb-4126-b05a-bf54951cb89e-kube-api-access-djfj5\") pod \"maas-api-key-cleanup-e2e-mt-c55c8ec1-29723970-l6rhc\" (UID: \"d6c1a612-14bb-4126-b05a-bf54951cb89e\") " pod="opendatahub/maas-api-key-cleanup-e2e-mt-c55c8ec1-29723970-l6rhc"
Jul 07 15:30:00.400929 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:00.400882    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djfj5\" (UniqueName: \"kubernetes.io/projected/d6c1a612-14bb-4126-b05a-bf54951cb89e-kube-api-access-djfj5\") pod \"maas-api-key-cleanup-e2e-mt-c55c8ec1-29723970-l6rhc\" (UID: \"d6c1a612-14bb-4126-b05a-bf54951cb89e\") " pod="opendatahub/maas-api-key-cleanup-e2e-mt-c55c8ec1-29723970-l6rhc"
Jul 07 15:30:00.417924 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:00.417815    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djfj5\" (UniqueName: \"kubernetes.io/projected/d6c1a612-14bb-4126-b05a-bf54951cb89e-kube-api-access-djfj5\") pod \"maas-api-key-cleanup-e2e-mt-c55c8ec1-29723970-l6rhc\" (UID: \"d6c1a612-14bb-4126-b05a-bf54951cb89e\") " pod="opendatahub/maas-api-key-cleanup-e2e-mt-c55c8ec1-29723970-l6rhc"
Jul 07 15:30:00.468751 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:00.468707    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-mt-c55c8ec1-29723970-l6rhc"
Jul 07 15:30:00.627067 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:00.627040    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-mt-c55c8ec1-29723970-l6rhc"]
Jul 07 15:30:00.628725 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:30:00.628692    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6c1a612_14bb_4126_b05a_bf54951cb89e.slice/crio-be2b70c8db0895118884c4cca4f6fe15055636c68cc38646f1fdabb784f92f85 WatchSource:0}: Error finding container be2b70c8db0895118884c4cca4f6fe15055636c68cc38646f1fdabb784f92f85: Status 404 returned error can't find the container with id be2b70c8db0895118884c4cca4f6fe15055636c68cc38646f1fdabb784f92f85
Jul 07 15:30:00.969425 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:00.969387    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-mt-c55c8ec1-29723970-l6rhc" event={"ID":"d6c1a612-14bb-4126-b05a-bf54951cb89e","Type":"ContainerStarted","Data":"f4d4e104f49992d0e38d53c5a4f680899065d187bea43c4a62e9bdcaba0865ad"}
Jul 07 15:30:00.969595 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:00.969475    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-mt-c55c8ec1-29723970-l6rhc" event={"ID":"d6c1a612-14bb-4126-b05a-bf54951cb89e","Type":"ContainerStarted","Data":"be2b70c8db0895118884c4cca4f6fe15055636c68cc38646f1fdabb784f92f85"}
Jul 07 15:30:00.988583 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:00.988533    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-mt-c55c8ec1-29723970-l6rhc" podStartSLOduration=0.988519473 podStartE2EDuration="988.519473ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:30:00.986621911 +0000 UTC m=+1768.783158639" watchObservedRunningTime="2026-07-07 15:30:00.988519473 +0000 UTC m=+1768.785056201"
Jul 07 15:30:01.976662 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:01.976580    2567 generic.go:356] "Generic (PLEG): container finished" podID="d6c1a612-14bb-4126-b05a-bf54951cb89e" containerID="f4d4e104f49992d0e38d53c5a4f680899065d187bea43c4a62e9bdcaba0865ad" exitCode=0
Jul 07 15:30:01.976662 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:01.976618    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-mt-c55c8ec1-29723970-l6rhc" event={"ID":"d6c1a612-14bb-4126-b05a-bf54951cb89e","Type":"ContainerDied","Data":"f4d4e104f49992d0e38d53c5a4f680899065d187bea43c4a62e9bdcaba0865ad"}
Jul 07 15:30:03.125353 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:03.125321    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-mt-c55c8ec1-29723970-l6rhc"
Jul 07 15:30:03.223552 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:03.223519    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djfj5\" (UniqueName: \"kubernetes.io/projected/d6c1a612-14bb-4126-b05a-bf54951cb89e-kube-api-access-djfj5\") pod \"d6c1a612-14bb-4126-b05a-bf54951cb89e\" (UID: \"d6c1a612-14bb-4126-b05a-bf54951cb89e\") "
Jul 07 15:30:03.226147 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:03.226112    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6c1a612-14bb-4126-b05a-bf54951cb89e-kube-api-access-djfj5" (OuterVolumeSpecName: "kube-api-access-djfj5") pod "d6c1a612-14bb-4126-b05a-bf54951cb89e" (UID: "d6c1a612-14bb-4126-b05a-bf54951cb89e"). InnerVolumeSpecName "kube-api-access-djfj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:30:03.324695 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:03.324655    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djfj5\" (UniqueName: \"kubernetes.io/projected/d6c1a612-14bb-4126-b05a-bf54951cb89e-kube-api-access-djfj5\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:30:03.987633 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:03.987604    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-mt-c55c8ec1-29723970-l6rhc"
Jul 07 15:30:03.987633 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:03.987621    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-mt-c55c8ec1-29723970-l6rhc" event={"ID":"d6c1a612-14bb-4126-b05a-bf54951cb89e","Type":"ContainerDied","Data":"be2b70c8db0895118884c4cca4f6fe15055636c68cc38646f1fdabb784f92f85"}
Jul 07 15:30:03.987859 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:03.987678    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="be2b70c8db0895118884c4cca4f6fe15055636c68cc38646f1fdabb784f92f85"
Jul 07 15:30:09.312289 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:09.312257    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph"]
Jul 07 15:30:09.312762 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:09.312469    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph" podUID="a4385c52-9a3c-4e29-9a81-9406c8fa54fe" containerName="maas-api" containerID="cri-o://ec32c4f3dc301e175b5bf0d8ee67516061727fe2c9557a5febdab3954c35bb02" gracePeriod=30
Jul 07 15:30:09.359578 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:09.359548    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-mt-c55c8ec1-29723970-l6rhc"]
Jul 07 15:30:09.361600 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:09.361572    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-mt-c55c8ec1-29723970-l6rhc"]
Jul 07 15:30:09.571412 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:09.571354    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph"
Jul 07 15:30:09.681180 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:09.681144    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjzsw\" (UniqueName: \"kubernetes.io/projected/a4385c52-9a3c-4e29-9a81-9406c8fa54fe-kube-api-access-bjzsw\") pod \"a4385c52-9a3c-4e29-9a81-9406c8fa54fe\" (UID: \"a4385c52-9a3c-4e29-9a81-9406c8fa54fe\") "
Jul 07 15:30:09.681381 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:09.681198    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4385c52-9a3c-4e29-9a81-9406c8fa54fe-maas-api-tls\") pod \"a4385c52-9a3c-4e29-9a81-9406c8fa54fe\" (UID: \"a4385c52-9a3c-4e29-9a81-9406c8fa54fe\") "
Jul 07 15:30:09.681381 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:09.681308    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4385c52-9a3c-4e29-9a81-9406c8fa54fe-openshift-service-ca\") pod \"a4385c52-9a3c-4e29-9a81-9406c8fa54fe\" (UID: \"a4385c52-9a3c-4e29-9a81-9406c8fa54fe\") "
Jul 07 15:30:09.681676 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:09.681655    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4385c52-9a3c-4e29-9a81-9406c8fa54fe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a4385c52-9a3c-4e29-9a81-9406c8fa54fe" (UID: "a4385c52-9a3c-4e29-9a81-9406c8fa54fe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:30:09.683433 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:09.683411    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4385c52-9a3c-4e29-9a81-9406c8fa54fe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a4385c52-9a3c-4e29-9a81-9406c8fa54fe" (UID: "a4385c52-9a3c-4e29-9a81-9406c8fa54fe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:30:09.683538 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:09.683466    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4385c52-9a3c-4e29-9a81-9406c8fa54fe-kube-api-access-bjzsw" (OuterVolumeSpecName: "kube-api-access-bjzsw") pod "a4385c52-9a3c-4e29-9a81-9406c8fa54fe" (UID: "a4385c52-9a3c-4e29-9a81-9406c8fa54fe"). InnerVolumeSpecName "kube-api-access-bjzsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:30:09.782896 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:09.782824    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4385c52-9a3c-4e29-9a81-9406c8fa54fe-openshift-service-ca\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:30:09.782896 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:09.782893    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjzsw\" (UniqueName: \"kubernetes.io/projected/a4385c52-9a3c-4e29-9a81-9406c8fa54fe-kube-api-access-bjzsw\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:30:09.782896 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:09.782910    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4385c52-9a3c-4e29-9a81-9406c8fa54fe-maas-api-tls\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:30:10.023790 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:10.023706    2567 generic.go:356] "Generic (PLEG): container finished" podID="a4385c52-9a3c-4e29-9a81-9406c8fa54fe" containerID="ec32c4f3dc301e175b5bf0d8ee67516061727fe2c9557a5febdab3954c35bb02" exitCode=0
Jul 07 15:30:10.023790 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:10.023752    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph"
Jul 07 15:30:10.023990 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:10.023799    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph" event={"ID":"a4385c52-9a3c-4e29-9a81-9406c8fa54fe","Type":"ContainerDied","Data":"ec32c4f3dc301e175b5bf0d8ee67516061727fe2c9557a5febdab3954c35bb02"}
Jul 07 15:30:10.023990 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:10.023876    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph" event={"ID":"a4385c52-9a3c-4e29-9a81-9406c8fa54fe","Type":"ContainerDied","Data":"647af8e511982fff81f9019bdb92b20af33fe3343600997e3ba402278462c84b"}
Jul 07 15:30:10.023990 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:10.023908    2567 scope.go:117] "RemoveContainer" containerID="ec32c4f3dc301e175b5bf0d8ee67516061727fe2c9557a5febdab3954c35bb02"
Jul 07 15:30:10.034091 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:10.034073    2567 scope.go:117] "RemoveContainer" containerID="ec32c4f3dc301e175b5bf0d8ee67516061727fe2c9557a5febdab3954c35bb02"
Jul 07 15:30:10.034387 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:30:10.034367    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec32c4f3dc301e175b5bf0d8ee67516061727fe2c9557a5febdab3954c35bb02\": container with ID starting with ec32c4f3dc301e175b5bf0d8ee67516061727fe2c9557a5febdab3954c35bb02 not found: ID does not exist" containerID="ec32c4f3dc301e175b5bf0d8ee67516061727fe2c9557a5febdab3954c35bb02"
Jul 07 15:30:10.034466 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:10.034396    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec32c4f3dc301e175b5bf0d8ee67516061727fe2c9557a5febdab3954c35bb02"} err="failed to get container status \"ec32c4f3dc301e175b5bf0d8ee67516061727fe2c9557a5febdab3954c35bb02\": rpc error: code = NotFound desc = could not find container \"ec32c4f3dc301e175b5bf0d8ee67516061727fe2c9557a5febdab3954c35bb02\": container with ID starting with ec32c4f3dc301e175b5bf0d8ee67516061727fe2c9557a5febdab3954c35bb02 not found: ID does not exist"
Jul 07 15:30:10.050378 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:10.050343    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph"]
Jul 07 15:30:10.054768 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:10.054740    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-c55c8ec1-5f957c968f-tpvph"]
Jul 07 15:30:10.777864 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:10.777802    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4385c52-9a3c-4e29-9a81-9406c8fa54fe" path="/var/lib/kubelet/pods/a4385c52-9a3c-4e29-9a81-9406c8fa54fe/volumes"
Jul 07 15:30:10.778239 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:10.778228    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6c1a612-14bb-4126-b05a-bf54951cb89e" path="/var/lib/kubelet/pods/d6c1a612-14bb-4126-b05a-bf54951cb89e/volumes"
Jul 07 15:30:32.814647 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:32.814615    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5r9rp_1c51cd93-b0ea-4aa1-8678-a4c833b2d37e/console-operator/1.log"
Jul 07 15:30:32.819584 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:30:32.819563    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5r9rp_1c51cd93-b0ea-4aa1-8678-a4c833b2d37e/console-operator/1.log"
Jul 07 15:31:04.012460 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:04.012420    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"]
Jul 07 15:31:04.013009 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:04.012710    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk" podUID="213670e0-be84-4c80-86e6-ab501e3b9d40" containerName="istio-proxy" containerID="cri-o://28fa8d8b1c1682fd1fce031278ad2b235d3d3e4964efedc8b1a73f94ec1b82ee" gracePeriod=30
Jul 07 15:31:07.565244 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:07.565193    2567 patch_prober.go:28] interesting pod/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 07 15:31:07.565625 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:07.565277    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk" podUID="213670e0-be84-4c80-86e6-ab501e3b9d40" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:31:09.272204 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.272137    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:31:09.342249 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.342211    2567 generic.go:356] "Generic (PLEG): container finished" podID="213670e0-be84-4c80-86e6-ab501e3b9d40" containerID="28fa8d8b1c1682fd1fce031278ad2b235d3d3e4964efedc8b1a73f94ec1b82ee" exitCode=0
Jul 07 15:31:09.342428 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.342267    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"
Jul 07 15:31:09.342428 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.342299    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk" event={"ID":"213670e0-be84-4c80-86e6-ab501e3b9d40","Type":"ContainerDied","Data":"28fa8d8b1c1682fd1fce031278ad2b235d3d3e4964efedc8b1a73f94ec1b82ee"}
Jul 07 15:31:09.342428 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.342347    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk" event={"ID":"213670e0-be84-4c80-86e6-ab501e3b9d40","Type":"ContainerDied","Data":"7028c39bf401b61b77d449c0052040efeb7a9e376eb7aff4a8a8764484eb1456"}
Jul 07 15:31:09.342428 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.342368    2567 scope.go:117] "RemoveContainer" containerID="28fa8d8b1c1682fd1fce031278ad2b235d3d3e4964efedc8b1a73f94ec1b82ee"
Jul 07 15:31:09.348658 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.348637    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-istio-data\") pod \"213670e0-be84-4c80-86e6-ab501e3b9d40\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") "
Jul 07 15:31:09.348821 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.348665    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/213670e0-be84-4c80-86e6-ab501e3b9d40-istio-token\") pod \"213670e0-be84-4c80-86e6-ab501e3b9d40\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") "
Jul 07 15:31:09.348821 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.348686    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-workload-certs\") pod \"213670e0-be84-4c80-86e6-ab501e3b9d40\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") "
Jul 07 15:31:09.348821 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.348740    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-workload-socket\") pod \"213670e0-be84-4c80-86e6-ab501e3b9d40\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") "
Jul 07 15:31:09.348821 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.348806    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/213670e0-be84-4c80-86e6-ab501e3b9d40-istio-podinfo\") pod \"213670e0-be84-4c80-86e6-ab501e3b9d40\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") "
Jul 07 15:31:09.349037 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.348867    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-credential-socket\") pod \"213670e0-be84-4c80-86e6-ab501e3b9d40\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") "
Jul 07 15:31:09.349037 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.348897    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-istio-envoy\") pod \"213670e0-be84-4c80-86e6-ab501e3b9d40\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") "
Jul 07 15:31:09.349037 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.348919    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/213670e0-be84-4c80-86e6-ab501e3b9d40-istiod-ca-cert\") pod \"213670e0-be84-4c80-86e6-ab501e3b9d40\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") "
Jul 07 15:31:09.349037 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.348987    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ww64v\" (UniqueName: \"kubernetes.io/projected/213670e0-be84-4c80-86e6-ab501e3b9d40-kube-api-access-ww64v\") pod \"213670e0-be84-4c80-86e6-ab501e3b9d40\" (UID: \"213670e0-be84-4c80-86e6-ab501e3b9d40\") "
Jul 07 15:31:09.349260 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.349114    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "213670e0-be84-4c80-86e6-ab501e3b9d40" (UID: "213670e0-be84-4c80-86e6-ab501e3b9d40"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:31:09.349367 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.349339    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-workload-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:31:09.349474 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.349451    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "213670e0-be84-4c80-86e6-ab501e3b9d40" (UID: "213670e0-be84-4c80-86e6-ab501e3b9d40"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:31:09.349577 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.349504    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-istio-data" (OuterVolumeSpecName: "istio-data") pod "213670e0-be84-4c80-86e6-ab501e3b9d40" (UID: "213670e0-be84-4c80-86e6-ab501e3b9d40"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:31:09.349656 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.349596    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/213670e0-be84-4c80-86e6-ab501e3b9d40-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "213670e0-be84-4c80-86e6-ab501e3b9d40" (UID: "213670e0-be84-4c80-86e6-ab501e3b9d40"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:31:09.349984 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.349953    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "213670e0-be84-4c80-86e6-ab501e3b9d40" (UID: "213670e0-be84-4c80-86e6-ab501e3b9d40"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:31:09.351096 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.351066    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/213670e0-be84-4c80-86e6-ab501e3b9d40-istio-token" (OuterVolumeSpecName: "istio-token") pod "213670e0-be84-4c80-86e6-ab501e3b9d40" (UID: "213670e0-be84-4c80-86e6-ab501e3b9d40"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:31:09.351466 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.351443    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "213670e0-be84-4c80-86e6-ab501e3b9d40" (UID: "213670e0-be84-4c80-86e6-ab501e3b9d40"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:31:09.351540 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.351506    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/213670e0-be84-4c80-86e6-ab501e3b9d40-kube-api-access-ww64v" (OuterVolumeSpecName: "kube-api-access-ww64v") pod "213670e0-be84-4c80-86e6-ab501e3b9d40" (UID: "213670e0-be84-4c80-86e6-ab501e3b9d40"). InnerVolumeSpecName "kube-api-access-ww64v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:31:09.351603 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.351585    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/213670e0-be84-4c80-86e6-ab501e3b9d40-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "213670e0-be84-4c80-86e6-ab501e3b9d40" (UID: "213670e0-be84-4c80-86e6-ab501e3b9d40"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 15:31:09.352755 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.352738    2567 scope.go:117] "RemoveContainer" containerID="5c287ead6a38f4c0a39ac08e23e87d2779798ef591de7540c08d70ff414aade7"
Jul 07 15:31:09.370931 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.370906    2567 scope.go:117] "RemoveContainer" containerID="28fa8d8b1c1682fd1fce031278ad2b235d3d3e4964efedc8b1a73f94ec1b82ee"
Jul 07 15:31:09.371250 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:31:09.371229    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28fa8d8b1c1682fd1fce031278ad2b235d3d3e4964efedc8b1a73f94ec1b82ee\": container with ID starting with 28fa8d8b1c1682fd1fce031278ad2b235d3d3e4964efedc8b1a73f94ec1b82ee not found: ID does not exist" containerID="28fa8d8b1c1682fd1fce031278ad2b235d3d3e4964efedc8b1a73f94ec1b82ee"
Jul 07 15:31:09.371307 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.371259    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28fa8d8b1c1682fd1fce031278ad2b235d3d3e4964efedc8b1a73f94ec1b82ee"} err="failed to get container status \"28fa8d8b1c1682fd1fce031278ad2b235d3d3e4964efedc8b1a73f94ec1b82ee\": rpc error: code = NotFound desc = could not find container \"28fa8d8b1c1682fd1fce031278ad2b235d3d3e4964efedc8b1a73f94ec1b82ee\": container with ID starting with 28fa8d8b1c1682fd1fce031278ad2b235d3d3e4964efedc8b1a73f94ec1b82ee not found: ID does not exist"
Jul 07 15:31:09.371307 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.371287    2567 scope.go:117] "RemoveContainer" containerID="5c287ead6a38f4c0a39ac08e23e87d2779798ef591de7540c08d70ff414aade7"
Jul 07 15:31:09.371535 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:31:09.371513    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c287ead6a38f4c0a39ac08e23e87d2779798ef591de7540c08d70ff414aade7\": container with ID starting with 5c287ead6a38f4c0a39ac08e23e87d2779798ef591de7540c08d70ff414aade7 not found: ID does not exist" containerID="5c287ead6a38f4c0a39ac08e23e87d2779798ef591de7540c08d70ff414aade7"
Jul 07 15:31:09.371603 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.371544    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c287ead6a38f4c0a39ac08e23e87d2779798ef591de7540c08d70ff414aade7"} err="failed to get container status \"5c287ead6a38f4c0a39ac08e23e87d2779798ef591de7540c08d70ff414aade7\": rpc error: code = NotFound desc = could not find container \"5c287ead6a38f4c0a39ac08e23e87d2779798ef591de7540c08d70ff414aade7\": container with ID starting with 5c287ead6a38f4c0a39ac08e23e87d2779798ef591de7540c08d70ff414aade7 not found: ID does not exist"
Jul 07 15:31:09.450348 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.450310    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-istio-envoy\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:31:09.450348 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.450342    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/213670e0-be84-4c80-86e6-ab501e3b9d40-istiod-ca-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:31:09.450348 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.450356    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ww64v\" (UniqueName: \"kubernetes.io/projected/213670e0-be84-4c80-86e6-ab501e3b9d40-kube-api-access-ww64v\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:31:09.450592 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.450370    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-istio-data\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:31:09.450592 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.450382    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/213670e0-be84-4c80-86e6-ab501e3b9d40-istio-token\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:31:09.450592 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.450393    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-workload-certs\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:31:09.450592 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.450403    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/213670e0-be84-4c80-86e6-ab501e3b9d40-istio-podinfo\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:31:09.450592 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.450412    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/213670e0-be84-4c80-86e6-ab501e3b9d40-credential-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:31:09.669591 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.669551    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"]
Jul 07 15:31:09.682795 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:09.682768    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-c55c8ec1-openshift-default-5f86ffdff-lfflk"]
Jul 07 15:31:10.777653 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:31:10.777619    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="213670e0-be84-4c80-86e6-ab501e3b9d40" path="/var/lib/kubelet/pods/213670e0-be84-4c80-86e6-ab501e3b9d40/volumes"
Jul 07 15:32:01.464604 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.464563    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"]
Jul 07 15:32:01.465183 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.465168    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="213670e0-be84-4c80-86e6-ab501e3b9d40" containerName="istio-proxy"
Jul 07 15:32:01.465231 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.465187    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="213670e0-be84-4c80-86e6-ab501e3b9d40" containerName="istio-proxy"
Jul 07 15:32:01.465231 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.465205    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="213670e0-be84-4c80-86e6-ab501e3b9d40" containerName="istio-proxy"
Jul 07 15:32:01.465231 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.465214    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="213670e0-be84-4c80-86e6-ab501e3b9d40" containerName="istio-proxy"
Jul 07 15:32:01.465332 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.465243    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4385c52-9a3c-4e29-9a81-9406c8fa54fe" containerName="maas-api"
Jul 07 15:32:01.465332 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.465253    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4385c52-9a3c-4e29-9a81-9406c8fa54fe" containerName="maas-api"
Jul 07 15:32:01.465332 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.465265    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6c1a612-14bb-4126-b05a-bf54951cb89e" containerName="cleanup"
Jul 07 15:32:01.465332 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.465273    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6c1a612-14bb-4126-b05a-bf54951cb89e" containerName="cleanup"
Jul 07 15:32:01.465454 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.465379    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4385c52-9a3c-4e29-9a81-9406c8fa54fe" containerName="maas-api"
Jul 07 15:32:01.465454 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.465390    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6c1a612-14bb-4126-b05a-bf54951cb89e" containerName="cleanup"
Jul 07 15:32:01.465454 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.465400    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="213670e0-be84-4c80-86e6-ab501e3b9d40" containerName="istio-proxy"
Jul 07 15:32:01.465454 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.465408    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="213670e0-be84-4c80-86e6-ab501e3b9d40" containerName="istio-proxy"
Jul 07 15:32:01.468934 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.468914    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.474001 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.473975    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-ac6910-openshift-default-dockercfg-b7mvn\""
Jul 07 15:32:01.481925 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.481901    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"]
Jul 07 15:32:01.511952 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.511913    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cd769be-ffd8-423f-b8af-c61ad201f61c-istio-token\") pod \"e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") " pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.512141 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.511964    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-credential-socket\") pod \"e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") " pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.512141 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.511991    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cd769be-ffd8-423f-b8af-c61ad201f61c-istiod-ca-cert\") pod \"e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") " pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.512141 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.512037    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-workload-socket\") pod \"e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") " pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.512141 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.512065    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cd769be-ffd8-423f-b8af-c61ad201f61c-istio-podinfo\") pod \"e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") " pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.512351 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.512144    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5j46\" (UniqueName: \"kubernetes.io/projected/7cd769be-ffd8-423f-b8af-c61ad201f61c-kube-api-access-f5j46\") pod \"e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") " pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.512351 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.512216    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-istio-envoy\") pod \"e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") " pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.512351 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.512260    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-istio-data\") pod \"e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") " pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.512351 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.512340    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-workload-certs\") pod \"e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") " pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.613206 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.613165    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5j46\" (UniqueName: \"kubernetes.io/projected/7cd769be-ffd8-423f-b8af-c61ad201f61c-kube-api-access-f5j46\") pod \"e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") " pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.613382 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.613224    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-istio-envoy\") pod \"e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") " pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.613382 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.613263    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-istio-data\") pod \"e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") " pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.613382 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.613317    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-workload-certs\") pod \"e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") " pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.613382 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.613348    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cd769be-ffd8-423f-b8af-c61ad201f61c-istio-token\") pod \"e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") " pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.613599 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.613403    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-credential-socket\") pod \"e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") " pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.613599 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.613520    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cd769be-ffd8-423f-b8af-c61ad201f61c-istiod-ca-cert\") pod \"e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") " pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.613599 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.613577    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-workload-socket\") pod \"e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") " pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.613760 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.613623    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cd769be-ffd8-423f-b8af-c61ad201f61c-istio-podinfo\") pod \"e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") " pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.613816 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.613758    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-istio-data\") pod \"e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") " pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.613816 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.613801    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-workload-certs\") pod \"e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") " pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.614168 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.614123    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-workload-socket\") pod \"e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") " pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.614344 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.614310    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cd769be-ffd8-423f-b8af-c61ad201f61c-istiod-ca-cert\") pod \"e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") " pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.614344 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.614322    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-credential-socket\") pod \"e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") " pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.616546 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.616333    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-istio-envoy\") pod \"e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") " pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.624918 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.616768    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cd769be-ffd8-423f-b8af-c61ad201f61c-istio-podinfo\") pod \"e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") " pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.624918 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.623255    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cd769be-ffd8-423f-b8af-c61ad201f61c-istio-token\") pod \"e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") " pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.625716 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.625688    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5j46\" (UniqueName: \"kubernetes.io/projected/7cd769be-ffd8-423f-b8af-c61ad201f61c-kube-api-access-f5j46\") pod \"e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") " pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.780554 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.780460    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:01.927816 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.927788    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"]
Jul 07 15:32:01.930607 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:32:01.930571    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cd769be_ffd8_423f_b8af_c61ad201f61c.slice/crio-cf5cce75984e4c6b84b31ca40303156ab87aaace7c7359de36e814c8ace2a005 WatchSource:0}: Error finding container cf5cce75984e4c6b84b31ca40303156ab87aaace7c7359de36e814c8ace2a005: Status 404 returned error can't find the container with id cf5cce75984e4c6b84b31ca40303156ab87aaace7c7359de36e814c8ace2a005
Jul 07 15:32:01.933030 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.932995    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:32:01.933110 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.933070    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:32:01.933110 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:01.933099    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:32:02.625899 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:02.625862    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95" event={"ID":"7cd769be-ffd8-423f-b8af-c61ad201f61c","Type":"ContainerStarted","Data":"37e3c13e687356b14866f97cb2dd74e7f639613f78676b5672d0a67b7868c169"}
Jul 07 15:32:02.626311 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:02.625930    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95" event={"ID":"7cd769be-ffd8-423f-b8af-c61ad201f61c","Type":"ContainerStarted","Data":"cf5cce75984e4c6b84b31ca40303156ab87aaace7c7359de36e814c8ace2a005"}
Jul 07 15:32:02.651999 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:02.651946    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95" podStartSLOduration=1.6519264219999998 podStartE2EDuration="1.651926422s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:32:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:32:02.64928643 +0000 UTC m=+1890.445823201" watchObservedRunningTime="2026-07-07 15:32:02.651926422 +0000 UTC m=+1890.448463150"
Jul 07 15:32:02.780665 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:02.780626    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:03.786397 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:03.786370    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:04.636982 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:04.636942    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:04.637865 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:04.637824    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:04.767098 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:04.767067    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf"]
Jul 07 15:32:04.769693 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:04.769678    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf"
Jul 07 15:32:04.775232 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:04.775209    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-ac6910-serving-cert\""
Jul 07 15:32:04.781444 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:04.781416    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf"]
Jul 07 15:32:04.848070 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:04.848033    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7affbc02-1ea4-4f65-8110-8e88cb6bfbef-maas-api-tls\") pod \"maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf\" (UID: \"7affbc02-1ea4-4f65-8110-8e88cb6bfbef\") " pod="opendatahub/maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf"
Jul 07 15:32:04.848553 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:04.848182    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zhzm\" (UniqueName: \"kubernetes.io/projected/7affbc02-1ea4-4f65-8110-8e88cb6bfbef-kube-api-access-6zhzm\") pod \"maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf\" (UID: \"7affbc02-1ea4-4f65-8110-8e88cb6bfbef\") " pod="opendatahub/maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf"
Jul 07 15:32:04.848553 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:04.848349    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7affbc02-1ea4-4f65-8110-8e88cb6bfbef-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf\" (UID: \"7affbc02-1ea4-4f65-8110-8e88cb6bfbef\") " pod="opendatahub/maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf"
Jul 07 15:32:04.949363 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:04.949282    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7affbc02-1ea4-4f65-8110-8e88cb6bfbef-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf\" (UID: \"7affbc02-1ea4-4f65-8110-8e88cb6bfbef\") " pod="opendatahub/maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf"
Jul 07 15:32:04.949363 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:04.949337    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7affbc02-1ea4-4f65-8110-8e88cb6bfbef-maas-api-tls\") pod \"maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf\" (UID: \"7affbc02-1ea4-4f65-8110-8e88cb6bfbef\") " pod="opendatahub/maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf"
Jul 07 15:32:04.949562 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:04.949392    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zhzm\" (UniqueName: \"kubernetes.io/projected/7affbc02-1ea4-4f65-8110-8e88cb6bfbef-kube-api-access-6zhzm\") pod \"maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf\" (UID: \"7affbc02-1ea4-4f65-8110-8e88cb6bfbef\") " pod="opendatahub/maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf"
Jul 07 15:32:04.949562 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:32:04.949512    2567 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-ac6910-serving-cert: secret "maas-api-e2e-inf-a-ac6910-serving-cert" not found
Jul 07 15:32:04.949635 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:32:04.949583    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7affbc02-1ea4-4f65-8110-8e88cb6bfbef-maas-api-tls podName:7affbc02-1ea4-4f65-8110-8e88cb6bfbef nodeName:}" failed. No retries permitted until 2026-07-07 15:32:05.449562494 +0000 UTC m=+1893.246099209 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7affbc02-1ea4-4f65-8110-8e88cb6bfbef-maas-api-tls") pod "maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf" (UID: "7affbc02-1ea4-4f65-8110-8e88cb6bfbef") : secret "maas-api-e2e-inf-a-ac6910-serving-cert" not found
Jul 07 15:32:04.950079 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:04.950059    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7affbc02-1ea4-4f65-8110-8e88cb6bfbef-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf\" (UID: \"7affbc02-1ea4-4f65-8110-8e88cb6bfbef\") " pod="opendatahub/maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf"
Jul 07 15:32:04.972347 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:04.972310    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zhzm\" (UniqueName: \"kubernetes.io/projected/7affbc02-1ea4-4f65-8110-8e88cb6bfbef-kube-api-access-6zhzm\") pod \"maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf\" (UID: \"7affbc02-1ea4-4f65-8110-8e88cb6bfbef\") " pod="opendatahub/maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf"
Jul 07 15:32:05.454845 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:05.454805    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7affbc02-1ea4-4f65-8110-8e88cb6bfbef-maas-api-tls\") pod \"maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf\" (UID: \"7affbc02-1ea4-4f65-8110-8e88cb6bfbef\") " pod="opendatahub/maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf"
Jul 07 15:32:05.457521 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:05.457496    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7affbc02-1ea4-4f65-8110-8e88cb6bfbef-maas-api-tls\") pod \"maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf\" (UID: \"7affbc02-1ea4-4f65-8110-8e88cb6bfbef\") " pod="opendatahub/maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf"
Jul 07 15:32:05.681593 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:05.681561    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf"
Jul 07 15:32:05.837290 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:05.837258    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf"]
Jul 07 15:32:05.839548 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:32:05.839522    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7affbc02_1ea4_4f65_8110_8e88cb6bfbef.slice/crio-aa6f33a4047f2e344bc30364828521e3255c334f6694a38b6098f8f54fe1073e WatchSource:0}: Error finding container aa6f33a4047f2e344bc30364828521e3255c334f6694a38b6098f8f54fe1073e: Status 404 returned error can't find the container with id aa6f33a4047f2e344bc30364828521e3255c334f6694a38b6098f8f54fe1073e
Jul 07 15:32:05.840969 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:05.840949    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 15:32:06.648161 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:06.648122    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf" event={"ID":"7affbc02-1ea4-4f65-8110-8e88cb6bfbef","Type":"ContainerStarted","Data":"aa6f33a4047f2e344bc30364828521e3255c334f6694a38b6098f8f54fe1073e"}
Jul 07 15:32:07.654668 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:07.654626    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf" event={"ID":"7affbc02-1ea4-4f65-8110-8e88cb6bfbef","Type":"ContainerStarted","Data":"f30255952b5dec12f38b52efac159b5185e86567b0fe76f0b942b0c1abcb0758"}
Jul 07 15:32:07.655139 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:07.654715    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf"
Jul 07 15:32:07.676169 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:07.676115    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf" podStartSLOduration=2.642622188 podStartE2EDuration="3.676099391s" totalImagesPullingTime="1.033477203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:32:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:32:07.674139504 +0000 UTC m=+1895.470676282" watchObservedRunningTime="2026-07-07 15:32:07.676099391 +0000 UTC m=+1895.472636118"
Jul 07 15:32:13.666053 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:13.666017    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf"
Jul 07 15:32:13.755910 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:13.742468    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m"]
Jul 07 15:32:13.757861 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:13.757799    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m"
Jul 07 15:32:13.763351 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:13.763317    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m"]
Jul 07 15:32:13.770008 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:13.769980    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-44430f-serving-cert\""
Jul 07 15:32:13.855486 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:13.855446    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98e17788-cb45-4b56-8896-0d419711bf34-maas-api-tls\") pod \"maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m\" (UID: \"98e17788-cb45-4b56-8896-0d419711bf34\") " pod="opendatahub/maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m"
Jul 07 15:32:13.855699 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:13.855508    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98e17788-cb45-4b56-8896-0d419711bf34-openshift-service-ca\") pod \"maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m\" (UID: \"98e17788-cb45-4b56-8896-0d419711bf34\") " pod="opendatahub/maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m"
Jul 07 15:32:13.855699 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:13.855527    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whsgr\" (UniqueName: \"kubernetes.io/projected/98e17788-cb45-4b56-8896-0d419711bf34-kube-api-access-whsgr\") pod \"maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m\" (UID: \"98e17788-cb45-4b56-8896-0d419711bf34\") " pod="opendatahub/maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m"
Jul 07 15:32:13.956704 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:13.956601    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98e17788-cb45-4b56-8896-0d419711bf34-maas-api-tls\") pod \"maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m\" (UID: \"98e17788-cb45-4b56-8896-0d419711bf34\") " pod="opendatahub/maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m"
Jul 07 15:32:13.956704 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:13.956693    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98e17788-cb45-4b56-8896-0d419711bf34-openshift-service-ca\") pod \"maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m\" (UID: \"98e17788-cb45-4b56-8896-0d419711bf34\") " pod="opendatahub/maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m"
Jul 07 15:32:13.957018 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:13.956876    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whsgr\" (UniqueName: \"kubernetes.io/projected/98e17788-cb45-4b56-8896-0d419711bf34-kube-api-access-whsgr\") pod \"maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m\" (UID: \"98e17788-cb45-4b56-8896-0d419711bf34\") " pod="opendatahub/maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m"
Jul 07 15:32:13.957322 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:13.957303    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98e17788-cb45-4b56-8896-0d419711bf34-openshift-service-ca\") pod \"maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m\" (UID: \"98e17788-cb45-4b56-8896-0d419711bf34\") " pod="opendatahub/maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m"
Jul 07 15:32:13.959473 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:13.959450    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98e17788-cb45-4b56-8896-0d419711bf34-maas-api-tls\") pod \"maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m\" (UID: \"98e17788-cb45-4b56-8896-0d419711bf34\") " pod="opendatahub/maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m"
Jul 07 15:32:13.969479 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:13.969454    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whsgr\" (UniqueName: \"kubernetes.io/projected/98e17788-cb45-4b56-8896-0d419711bf34-kube-api-access-whsgr\") pod \"maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m\" (UID: \"98e17788-cb45-4b56-8896-0d419711bf34\") " pod="opendatahub/maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m"
Jul 07 15:32:14.075210 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:14.075177    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m"
Jul 07 15:32:14.285448 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:14.285384    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m"]
Jul 07 15:32:14.286908 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:32:14.286885    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98e17788_cb45_4b56_8896_0d419711bf34.slice/crio-e060a34075917b6cc571d777ebc1c338c6325ee68938cbd9e1335a27f83e8661 WatchSource:0}: Error finding container e060a34075917b6cc571d777ebc1c338c6325ee68938cbd9e1335a27f83e8661: Status 404 returned error can't find the container with id e060a34075917b6cc571d777ebc1c338c6325ee68938cbd9e1335a27f83e8661
Jul 07 15:32:14.696053 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:14.696015    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m" event={"ID":"98e17788-cb45-4b56-8896-0d419711bf34","Type":"ContainerStarted","Data":"e060a34075917b6cc571d777ebc1c338c6325ee68938cbd9e1335a27f83e8661"}
Jul 07 15:32:15.703180 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:15.703141    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m" event={"ID":"98e17788-cb45-4b56-8896-0d419711bf34","Type":"ContainerStarted","Data":"119e1f4a1fda9cebce957641dcb760898a5cdc6dfb4d2ac85d48114a476f1d97"}
Jul 07 15:32:15.703581 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:15.703484    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m"
Jul 07 15:32:15.727003 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:15.726920    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m" podStartSLOduration=2.244444503 podStartE2EDuration="2.726883389s" totalImagesPullingTime="482.438886ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:32:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:32:15.725146663 +0000 UTC m=+1903.521683392" watchObservedRunningTime="2026-07-07 15:32:15.726883389 +0000 UTC m=+1903.523420118"
Jul 07 15:32:20.048297 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:20.048265    2567 patch_prober.go:28] interesting pod/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": read tcp 10.132.0.2:57978->10.132.0.70:15021: read: connection reset by peer" start-of-body=
Jul 07 15:32:20.048679 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:20.048336    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95" podUID="7cd769be-ffd8-423f-b8af-c61ad201f61c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": read tcp 10.132.0.2:57978->10.132.0.70:15021: read: connection reset by peer"
Jul 07 15:32:20.735085 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:20.735046    2567 generic.go:356] "Generic (PLEG): container finished" podID="7cd769be-ffd8-423f-b8af-c61ad201f61c" containerID="37e3c13e687356b14866f97cb2dd74e7f639613f78676b5672d0a67b7868c169" exitCode=0
Jul 07 15:32:20.735272 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:20.735114    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95" event={"ID":"7cd769be-ffd8-423f-b8af-c61ad201f61c","Type":"ContainerDied","Data":"37e3c13e687356b14866f97cb2dd74e7f639613f78676b5672d0a67b7868c169"}
Jul 07 15:32:20.735584 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:20.735565    2567 scope.go:117] "RemoveContainer" containerID="37e3c13e687356b14866f97cb2dd74e7f639613f78676b5672d0a67b7868c169"
Jul 07 15:32:20.736597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:20.736571    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:32:20.736655 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:20.736638    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:32:20.736700 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:20.736670    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:32:21.714286 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:21.714253    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m"
Jul 07 15:32:21.742505 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:21.742463    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95" event={"ID":"7cd769be-ffd8-423f-b8af-c61ad201f61c","Type":"ContainerStarted","Data":"7671aae1e086666d093ea6430d1b338416c02f129138a2f7a83012f0cc2c5e26"}
Jul 07 15:32:21.780598 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:21.780568    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:21.780973 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:21.780950    2567 patch_prober.go:28] interesting pod/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Jul 07 15:32:21.781036 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:21.781006    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95" podUID="7cd769be-ffd8-423f-b8af-c61ad201f61c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Jul 07 15:32:22.781369 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:22.781330    2567 patch_prober.go:28] interesting pod/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Jul 07 15:32:22.781775 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:22.781383    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95" podUID="7cd769be-ffd8-423f-b8af-c61ad201f61c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Jul 07 15:32:23.781384 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:23.781340    2567 patch_prober.go:28] interesting pod/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Jul 07 15:32:23.781939 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:23.781403    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95" podUID="7cd769be-ffd8-423f-b8af-c61ad201f61c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Jul 07 15:32:24.781251 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:24.781213    2567 patch_prober.go:28] interesting pod/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Jul 07 15:32:24.781430 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:24.781272    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95" podUID="7cd769be-ffd8-423f-b8af-c61ad201f61c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Jul 07 15:32:25.781742 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:25.781704    2567 patch_prober.go:28] interesting pod/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Jul 07 15:32:25.782185 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:25.781758    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95" podUID="7cd769be-ffd8-423f-b8af-c61ad201f61c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Jul 07 15:32:26.784564 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:26.784532    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:26.785044 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:26.784940    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:32:26.785768 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:32:26.785753    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:33:19.196682 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:33:19.196649    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf"]
Jul 07 15:33:19.197681 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:33:19.196900    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf" podUID="7affbc02-1ea4-4f65-8110-8e88cb6bfbef" containerName="maas-api" containerID="cri-o://f30255952b5dec12f38b52efac159b5185e86567b0fe76f0b942b0c1abcb0758" gracePeriod=30
Jul 07 15:33:19.462724 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:33:19.462662    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf"
Jul 07 15:33:19.577081 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:33:19.577044    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zhzm\" (UniqueName: \"kubernetes.io/projected/7affbc02-1ea4-4f65-8110-8e88cb6bfbef-kube-api-access-6zhzm\") pod \"7affbc02-1ea4-4f65-8110-8e88cb6bfbef\" (UID: \"7affbc02-1ea4-4f65-8110-8e88cb6bfbef\") "
Jul 07 15:33:19.577297 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:33:19.577087    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7affbc02-1ea4-4f65-8110-8e88cb6bfbef-maas-api-tls\") pod \"7affbc02-1ea4-4f65-8110-8e88cb6bfbef\" (UID: \"7affbc02-1ea4-4f65-8110-8e88cb6bfbef\") "
Jul 07 15:33:19.577297 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:33:19.577168    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7affbc02-1ea4-4f65-8110-8e88cb6bfbef-openshift-service-ca\") pod \"7affbc02-1ea4-4f65-8110-8e88cb6bfbef\" (UID: \"7affbc02-1ea4-4f65-8110-8e88cb6bfbef\") "
Jul 07 15:33:19.577561 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:33:19.577538    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7affbc02-1ea4-4f65-8110-8e88cb6bfbef-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7affbc02-1ea4-4f65-8110-8e88cb6bfbef" (UID: "7affbc02-1ea4-4f65-8110-8e88cb6bfbef"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:33:19.579371 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:33:19.579332    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7affbc02-1ea4-4f65-8110-8e88cb6bfbef-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7affbc02-1ea4-4f65-8110-8e88cb6bfbef" (UID: "7affbc02-1ea4-4f65-8110-8e88cb6bfbef"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:33:19.579486 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:33:19.579462    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7affbc02-1ea4-4f65-8110-8e88cb6bfbef-kube-api-access-6zhzm" (OuterVolumeSpecName: "kube-api-access-6zhzm") pod "7affbc02-1ea4-4f65-8110-8e88cb6bfbef" (UID: "7affbc02-1ea4-4f65-8110-8e88cb6bfbef"). InnerVolumeSpecName "kube-api-access-6zhzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:33:19.678362 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:33:19.678326    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zhzm\" (UniqueName: \"kubernetes.io/projected/7affbc02-1ea4-4f65-8110-8e88cb6bfbef-kube-api-access-6zhzm\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:33:19.678362 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:33:19.678358    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7affbc02-1ea4-4f65-8110-8e88cb6bfbef-maas-api-tls\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:33:19.678362 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:33:19.678367    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7affbc02-1ea4-4f65-8110-8e88cb6bfbef-openshift-service-ca\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:33:20.065051 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:33:20.065016    2567 generic.go:356] "Generic (PLEG): container finished" podID="7affbc02-1ea4-4f65-8110-8e88cb6bfbef" containerID="f30255952b5dec12f38b52efac159b5185e86567b0fe76f0b942b0c1abcb0758" exitCode=0
Jul 07 15:33:20.065224 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:33:20.065062    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf"
Jul 07 15:33:20.065224 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:33:20.065098    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf" event={"ID":"7affbc02-1ea4-4f65-8110-8e88cb6bfbef","Type":"ContainerDied","Data":"f30255952b5dec12f38b52efac159b5185e86567b0fe76f0b942b0c1abcb0758"}
Jul 07 15:33:20.065224 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:33:20.065147    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf" event={"ID":"7affbc02-1ea4-4f65-8110-8e88cb6bfbef","Type":"ContainerDied","Data":"aa6f33a4047f2e344bc30364828521e3255c334f6694a38b6098f8f54fe1073e"}
Jul 07 15:33:20.065224 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:33:20.065170    2567 scope.go:117] "RemoveContainer" containerID="f30255952b5dec12f38b52efac159b5185e86567b0fe76f0b942b0c1abcb0758"
Jul 07 15:33:20.075618 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:33:20.075596    2567 scope.go:117] "RemoveContainer" containerID="f30255952b5dec12f38b52efac159b5185e86567b0fe76f0b942b0c1abcb0758"
Jul 07 15:33:20.075902 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:33:20.075882    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f30255952b5dec12f38b52efac159b5185e86567b0fe76f0b942b0c1abcb0758\": container with ID starting with f30255952b5dec12f38b52efac159b5185e86567b0fe76f0b942b0c1abcb0758 not found: ID does not exist" containerID="f30255952b5dec12f38b52efac159b5185e86567b0fe76f0b942b0c1abcb0758"
Jul 07 15:33:20.076221 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:33:20.075910    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f30255952b5dec12f38b52efac159b5185e86567b0fe76f0b942b0c1abcb0758"} err="failed to get container status \"f30255952b5dec12f38b52efac159b5185e86567b0fe76f0b942b0c1abcb0758\": rpc error: code = NotFound desc = could not find container \"f30255952b5dec12f38b52efac159b5185e86567b0fe76f0b942b0c1abcb0758\": container with ID starting with f30255952b5dec12f38b52efac159b5185e86567b0fe76f0b942b0c1abcb0758 not found: ID does not exist"
Jul 07 15:33:20.089246 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:33:20.089222    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf"]
Jul 07 15:33:20.094272 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:33:20.094247    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-ac6910-89bf6cf76-vdxzf"]
Jul 07 15:33:20.778335 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:33:20.778303    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7affbc02-1ea4-4f65-8110-8e88cb6bfbef" path="/var/lib/kubelet/pods/7affbc02-1ea4-4f65-8110-8e88cb6bfbef/volumes"
Jul 07 15:34:01.626008 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:01.625961    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"]
Jul 07 15:34:01.626521 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:01.626231    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95" podUID="7cd769be-ffd8-423f-b8af-c61ad201f61c" containerName="istio-proxy" containerID="cri-o://7671aae1e086666d093ea6430d1b338416c02f129138a2f7a83012f0cc2c5e26" gracePeriod=30
Jul 07 15:34:02.994605 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:02.992126    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m"]
Jul 07 15:34:02.994605 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:02.992474    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m" podUID="98e17788-cb45-4b56-8896-0d419711bf34" containerName="maas-api" containerID="cri-o://119e1f4a1fda9cebce957641dcb760898a5cdc6dfb4d2ac85d48114a476f1d97" gracePeriod=30
Jul 07 15:34:03.258172 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:03.258110    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m"
Jul 07 15:34:03.300195 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:03.300158    2567 generic.go:356] "Generic (PLEG): container finished" podID="98e17788-cb45-4b56-8896-0d419711bf34" containerID="119e1f4a1fda9cebce957641dcb760898a5cdc6dfb4d2ac85d48114a476f1d97" exitCode=0
Jul 07 15:34:03.300451 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:03.300214    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m"
Jul 07 15:34:03.300589 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:03.300198    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m" event={"ID":"98e17788-cb45-4b56-8896-0d419711bf34","Type":"ContainerDied","Data":"119e1f4a1fda9cebce957641dcb760898a5cdc6dfb4d2ac85d48114a476f1d97"}
Jul 07 15:34:03.300662 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:03.300597    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m" event={"ID":"98e17788-cb45-4b56-8896-0d419711bf34","Type":"ContainerDied","Data":"e060a34075917b6cc571d777ebc1c338c6325ee68938cbd9e1335a27f83e8661"}
Jul 07 15:34:03.300662 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:03.300636    2567 scope.go:117] "RemoveContainer" containerID="119e1f4a1fda9cebce957641dcb760898a5cdc6dfb4d2ac85d48114a476f1d97"
Jul 07 15:34:03.310309 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:03.310288    2567 scope.go:117] "RemoveContainer" containerID="119e1f4a1fda9cebce957641dcb760898a5cdc6dfb4d2ac85d48114a476f1d97"
Jul 07 15:34:03.310626 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:34:03.310587    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"119e1f4a1fda9cebce957641dcb760898a5cdc6dfb4d2ac85d48114a476f1d97\": container with ID starting with 119e1f4a1fda9cebce957641dcb760898a5cdc6dfb4d2ac85d48114a476f1d97 not found: ID does not exist" containerID="119e1f4a1fda9cebce957641dcb760898a5cdc6dfb4d2ac85d48114a476f1d97"
Jul 07 15:34:03.310674 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:03.310639    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"119e1f4a1fda9cebce957641dcb760898a5cdc6dfb4d2ac85d48114a476f1d97"} err="failed to get container status \"119e1f4a1fda9cebce957641dcb760898a5cdc6dfb4d2ac85d48114a476f1d97\": rpc error: code = NotFound desc = could not find container \"119e1f4a1fda9cebce957641dcb760898a5cdc6dfb4d2ac85d48114a476f1d97\": container with ID starting with 119e1f4a1fda9cebce957641dcb760898a5cdc6dfb4d2ac85d48114a476f1d97 not found: ID does not exist"
Jul 07 15:34:03.383185 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:03.383148    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whsgr\" (UniqueName: \"kubernetes.io/projected/98e17788-cb45-4b56-8896-0d419711bf34-kube-api-access-whsgr\") pod \"98e17788-cb45-4b56-8896-0d419711bf34\" (UID: \"98e17788-cb45-4b56-8896-0d419711bf34\") "
Jul 07 15:34:03.383355 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:03.383204    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98e17788-cb45-4b56-8896-0d419711bf34-openshift-service-ca\") pod \"98e17788-cb45-4b56-8896-0d419711bf34\" (UID: \"98e17788-cb45-4b56-8896-0d419711bf34\") "
Jul 07 15:34:03.383355 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:03.383243    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98e17788-cb45-4b56-8896-0d419711bf34-maas-api-tls\") pod \"98e17788-cb45-4b56-8896-0d419711bf34\" (UID: \"98e17788-cb45-4b56-8896-0d419711bf34\") "
Jul 07 15:34:03.383608 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:03.383583    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98e17788-cb45-4b56-8896-0d419711bf34-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "98e17788-cb45-4b56-8896-0d419711bf34" (UID: "98e17788-cb45-4b56-8896-0d419711bf34"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:34:03.385384 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:03.385360    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98e17788-cb45-4b56-8896-0d419711bf34-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "98e17788-cb45-4b56-8896-0d419711bf34" (UID: "98e17788-cb45-4b56-8896-0d419711bf34"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:34:03.385479 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:03.385387    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98e17788-cb45-4b56-8896-0d419711bf34-kube-api-access-whsgr" (OuterVolumeSpecName: "kube-api-access-whsgr") pod "98e17788-cb45-4b56-8896-0d419711bf34" (UID: "98e17788-cb45-4b56-8896-0d419711bf34"). InnerVolumeSpecName "kube-api-access-whsgr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:34:03.484440 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:03.484400    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whsgr\" (UniqueName: \"kubernetes.io/projected/98e17788-cb45-4b56-8896-0d419711bf34-kube-api-access-whsgr\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:34:03.484440 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:03.484433    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98e17788-cb45-4b56-8896-0d419711bf34-openshift-service-ca\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:34:03.484657 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:03.484450    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98e17788-cb45-4b56-8896-0d419711bf34-maas-api-tls\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:34:03.627652 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:03.627622    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m"]
Jul 07 15:34:03.632021 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:03.631995    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-44430f-6f7d94599b-nxd7m"]
Jul 07 15:34:04.779066 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:04.779030    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98e17788-cb45-4b56-8896-0d419711bf34" path="/var/lib/kubelet/pods/98e17788-cb45-4b56-8896-0d419711bf34/volumes"
Jul 07 15:34:06.891044 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:06.890984    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:34:07.013610 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.013579    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cd769be-ffd8-423f-b8af-c61ad201f61c-istio-token\") pod \"7cd769be-ffd8-423f-b8af-c61ad201f61c\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") "
Jul 07 15:34:07.013823 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.013657    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cd769be-ffd8-423f-b8af-c61ad201f61c-istio-podinfo\") pod \"7cd769be-ffd8-423f-b8af-c61ad201f61c\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") "
Jul 07 15:34:07.013823 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.013688    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-istio-data\") pod \"7cd769be-ffd8-423f-b8af-c61ad201f61c\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") "
Jul 07 15:34:07.013823 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.013747    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-credential-socket\") pod \"7cd769be-ffd8-423f-b8af-c61ad201f61c\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") "
Jul 07 15:34:07.013823 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.013771    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cd769be-ffd8-423f-b8af-c61ad201f61c-istiod-ca-cert\") pod \"7cd769be-ffd8-423f-b8af-c61ad201f61c\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") "
Jul 07 15:34:07.013823 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.013814    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-workload-socket\") pod \"7cd769be-ffd8-423f-b8af-c61ad201f61c\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") "
Jul 07 15:34:07.014148 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.013875    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f5j46\" (UniqueName: \"kubernetes.io/projected/7cd769be-ffd8-423f-b8af-c61ad201f61c-kube-api-access-f5j46\") pod \"7cd769be-ffd8-423f-b8af-c61ad201f61c\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") "
Jul 07 15:34:07.014148 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.013913    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-workload-certs\") pod \"7cd769be-ffd8-423f-b8af-c61ad201f61c\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") "
Jul 07 15:34:07.014148 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.013969    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-istio-envoy\") pod \"7cd769be-ffd8-423f-b8af-c61ad201f61c\" (UID: \"7cd769be-ffd8-423f-b8af-c61ad201f61c\") "
Jul 07 15:34:07.014148 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.014090    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7cd769be-ffd8-423f-b8af-c61ad201f61c" (UID: "7cd769be-ffd8-423f-b8af-c61ad201f61c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:34:07.014364 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.014159    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7cd769be-ffd8-423f-b8af-c61ad201f61c" (UID: "7cd769be-ffd8-423f-b8af-c61ad201f61c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:34:07.014364 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.014295    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-workload-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:34:07.014364 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.014313    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-credential-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:34:07.014597 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.014576    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cd769be-ffd8-423f-b8af-c61ad201f61c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7cd769be-ffd8-423f-b8af-c61ad201f61c" (UID: "7cd769be-ffd8-423f-b8af-c61ad201f61c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:34:07.014770 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.014717    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-istio-data" (OuterVolumeSpecName: "istio-data") pod "7cd769be-ffd8-423f-b8af-c61ad201f61c" (UID: "7cd769be-ffd8-423f-b8af-c61ad201f61c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:34:07.015004 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.014983    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7cd769be-ffd8-423f-b8af-c61ad201f61c" (UID: "7cd769be-ffd8-423f-b8af-c61ad201f61c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:34:07.016419 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.016396    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cd769be-ffd8-423f-b8af-c61ad201f61c-istio-token" (OuterVolumeSpecName: "istio-token") pod "7cd769be-ffd8-423f-b8af-c61ad201f61c" (UID: "7cd769be-ffd8-423f-b8af-c61ad201f61c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:34:07.016693 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.016671    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cd769be-ffd8-423f-b8af-c61ad201f61c-kube-api-access-f5j46" (OuterVolumeSpecName: "kube-api-access-f5j46") pod "7cd769be-ffd8-423f-b8af-c61ad201f61c" (UID: "7cd769be-ffd8-423f-b8af-c61ad201f61c"). InnerVolumeSpecName "kube-api-access-f5j46". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:34:07.016851 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.016812    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7cd769be-ffd8-423f-b8af-c61ad201f61c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7cd769be-ffd8-423f-b8af-c61ad201f61c" (UID: "7cd769be-ffd8-423f-b8af-c61ad201f61c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 15:34:07.016925 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.016853    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7cd769be-ffd8-423f-b8af-c61ad201f61c" (UID: "7cd769be-ffd8-423f-b8af-c61ad201f61c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:34:07.114905 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.114866    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cd769be-ffd8-423f-b8af-c61ad201f61c-istiod-ca-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:34:07.114905 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.114898    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5j46\" (UniqueName: \"kubernetes.io/projected/7cd769be-ffd8-423f-b8af-c61ad201f61c-kube-api-access-f5j46\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:34:07.114905 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.114912    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-workload-certs\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:34:07.115144 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.114922    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-istio-envoy\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:34:07.115144 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.114930    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cd769be-ffd8-423f-b8af-c61ad201f61c-istio-token\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:34:07.115144 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.114937    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cd769be-ffd8-423f-b8af-c61ad201f61c-istio-podinfo\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:34:07.115144 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.114946    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cd769be-ffd8-423f-b8af-c61ad201f61c-istio-data\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:34:07.324695 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.324662    2567 generic.go:356] "Generic (PLEG): container finished" podID="7cd769be-ffd8-423f-b8af-c61ad201f61c" containerID="7671aae1e086666d093ea6430d1b338416c02f129138a2f7a83012f0cc2c5e26" exitCode=0
Jul 07 15:34:07.324896 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.324719    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"
Jul 07 15:34:07.324896 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.324757    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95" event={"ID":"7cd769be-ffd8-423f-b8af-c61ad201f61c","Type":"ContainerDied","Data":"7671aae1e086666d093ea6430d1b338416c02f129138a2f7a83012f0cc2c5e26"}
Jul 07 15:34:07.324896 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.324814    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95" event={"ID":"7cd769be-ffd8-423f-b8af-c61ad201f61c","Type":"ContainerDied","Data":"cf5cce75984e4c6b84b31ca40303156ab87aaace7c7359de36e814c8ace2a005"}
Jul 07 15:34:07.324896 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.324858    2567 scope.go:117] "RemoveContainer" containerID="7671aae1e086666d093ea6430d1b338416c02f129138a2f7a83012f0cc2c5e26"
Jul 07 15:34:07.335139 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.335120    2567 scope.go:117] "RemoveContainer" containerID="37e3c13e687356b14866f97cb2dd74e7f639613f78676b5672d0a67b7868c169"
Jul 07 15:34:07.343595 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.343571    2567 scope.go:117] "RemoveContainer" containerID="7671aae1e086666d093ea6430d1b338416c02f129138a2f7a83012f0cc2c5e26"
Jul 07 15:34:07.343871 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:34:07.343849    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7671aae1e086666d093ea6430d1b338416c02f129138a2f7a83012f0cc2c5e26\": container with ID starting with 7671aae1e086666d093ea6430d1b338416c02f129138a2f7a83012f0cc2c5e26 not found: ID does not exist" containerID="7671aae1e086666d093ea6430d1b338416c02f129138a2f7a83012f0cc2c5e26"
Jul 07 15:34:07.343951 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.343880    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7671aae1e086666d093ea6430d1b338416c02f129138a2f7a83012f0cc2c5e26"} err="failed to get container status \"7671aae1e086666d093ea6430d1b338416c02f129138a2f7a83012f0cc2c5e26\": rpc error: code = NotFound desc = could not find container \"7671aae1e086666d093ea6430d1b338416c02f129138a2f7a83012f0cc2c5e26\": container with ID starting with 7671aae1e086666d093ea6430d1b338416c02f129138a2f7a83012f0cc2c5e26 not found: ID does not exist"
Jul 07 15:34:07.343951 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.343898    2567 scope.go:117] "RemoveContainer" containerID="37e3c13e687356b14866f97cb2dd74e7f639613f78676b5672d0a67b7868c169"
Jul 07 15:34:07.344148 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:34:07.344133    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37e3c13e687356b14866f97cb2dd74e7f639613f78676b5672d0a67b7868c169\": container with ID starting with 37e3c13e687356b14866f97cb2dd74e7f639613f78676b5672d0a67b7868c169 not found: ID does not exist" containerID="37e3c13e687356b14866f97cb2dd74e7f639613f78676b5672d0a67b7868c169"
Jul 07 15:34:07.344193 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.344152    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37e3c13e687356b14866f97cb2dd74e7f639613f78676b5672d0a67b7868c169"} err="failed to get container status \"37e3c13e687356b14866f97cb2dd74e7f639613f78676b5672d0a67b7868c169\": rpc error: code = NotFound desc = could not find container \"37e3c13e687356b14866f97cb2dd74e7f639613f78676b5672d0a67b7868c169\": container with ID starting with 37e3c13e687356b14866f97cb2dd74e7f639613f78676b5672d0a67b7868c169 not found: ID does not exist"
Jul 07 15:34:07.349855 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.349814    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"]
Jul 07 15:34:07.356588 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:07.356568    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-ac6910-openshift-default-7d7f97cdcc-wph95"]
Jul 07 15:34:08.778544 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:08.778496    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cd769be-ffd8-423f-b8af-c61ad201f61c" path="/var/lib/kubelet/pods/7cd769be-ffd8-423f-b8af-c61ad201f61c/volumes"
Jul 07 15:34:10.649141 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.649106    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"]
Jul 07 15:34:10.649602 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.649558    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98e17788-cb45-4b56-8896-0d419711bf34" containerName="maas-api"
Jul 07 15:34:10.649602 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.649570    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="98e17788-cb45-4b56-8896-0d419711bf34" containerName="maas-api"
Jul 07 15:34:10.649602 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.649585    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cd769be-ffd8-423f-b8af-c61ad201f61c" containerName="istio-proxy"
Jul 07 15:34:10.649602 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.649592    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cd769be-ffd8-423f-b8af-c61ad201f61c" containerName="istio-proxy"
Jul 07 15:34:10.649826 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.649605    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7affbc02-1ea4-4f65-8110-8e88cb6bfbef" containerName="maas-api"
Jul 07 15:34:10.649826 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.649612    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7affbc02-1ea4-4f65-8110-8e88cb6bfbef" containerName="maas-api"
Jul 07 15:34:10.649826 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.649620    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cd769be-ffd8-423f-b8af-c61ad201f61c" containerName="istio-proxy"
Jul 07 15:34:10.649826 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.649625    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cd769be-ffd8-423f-b8af-c61ad201f61c" containerName="istio-proxy"
Jul 07 15:34:10.649826 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.649695    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cd769be-ffd8-423f-b8af-c61ad201f61c" containerName="istio-proxy"
Jul 07 15:34:10.649826 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.649709    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7affbc02-1ea4-4f65-8110-8e88cb6bfbef" containerName="maas-api"
Jul 07 15:34:10.649826 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.649714    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="98e17788-cb45-4b56-8896-0d419711bf34" containerName="maas-api"
Jul 07 15:34:10.654328 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.654304    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:10.657227 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.657205    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-51978c-openshift-default-dockercfg-xvthv\""
Jul 07 15:34:10.664817 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.664789    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"]
Jul 07 15:34:10.750209 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.750161    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4297409-0821-412c-a8da-ac4f328b34e0-istio-podinfo\") pod \"e2e-api-a-51978c-openshift-default-757789f697-8q844\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") " pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:10.750209 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.750206    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4297409-0821-412c-a8da-ac4f328b34e0-istiod-ca-cert\") pod \"e2e-api-a-51978c-openshift-default-757789f697-8q844\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") " pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:10.750429 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.750293    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-istio-data\") pod \"e2e-api-a-51978c-openshift-default-757789f697-8q844\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") " pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:10.750429 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.750328    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5sdf\" (UniqueName: \"kubernetes.io/projected/e4297409-0821-412c-a8da-ac4f328b34e0-kube-api-access-p5sdf\") pod \"e2e-api-a-51978c-openshift-default-757789f697-8q844\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") " pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:10.750429 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.750359    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-workload-certs\") pod \"e2e-api-a-51978c-openshift-default-757789f697-8q844\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") " pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:10.750429 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.750389    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-workload-socket\") pod \"e2e-api-a-51978c-openshift-default-757789f697-8q844\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") " pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:10.750590 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.750466    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-credential-socket\") pod \"e2e-api-a-51978c-openshift-default-757789f697-8q844\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") " pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:10.750590 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.750532    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4297409-0821-412c-a8da-ac4f328b34e0-istio-token\") pod \"e2e-api-a-51978c-openshift-default-757789f697-8q844\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") " pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:10.750590 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.750561    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-istio-envoy\") pod \"e2e-api-a-51978c-openshift-default-757789f697-8q844\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") " pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:10.851525 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.851485    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-credential-socket\") pod \"e2e-api-a-51978c-openshift-default-757789f697-8q844\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") " pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:10.851710 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.851542    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4297409-0821-412c-a8da-ac4f328b34e0-istio-token\") pod \"e2e-api-a-51978c-openshift-default-757789f697-8q844\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") " pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:10.851710 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.851562    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-istio-envoy\") pod \"e2e-api-a-51978c-openshift-default-757789f697-8q844\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") " pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:10.851710 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.851585    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4297409-0821-412c-a8da-ac4f328b34e0-istio-podinfo\") pod \"e2e-api-a-51978c-openshift-default-757789f697-8q844\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") " pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:10.851710 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.851601    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4297409-0821-412c-a8da-ac4f328b34e0-istiod-ca-cert\") pod \"e2e-api-a-51978c-openshift-default-757789f697-8q844\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") " pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:10.851710 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.851629    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-istio-data\") pod \"e2e-api-a-51978c-openshift-default-757789f697-8q844\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") " pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:10.851710 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.851655    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5sdf\" (UniqueName: \"kubernetes.io/projected/e4297409-0821-412c-a8da-ac4f328b34e0-kube-api-access-p5sdf\") pod \"e2e-api-a-51978c-openshift-default-757789f697-8q844\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") " pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:10.851710 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.851685    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-workload-certs\") pod \"e2e-api-a-51978c-openshift-default-757789f697-8q844\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") " pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:10.852118 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.851714    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-workload-socket\") pod \"e2e-api-a-51978c-openshift-default-757789f697-8q844\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") " pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:10.852118 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.851971    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-credential-socket\") pod \"e2e-api-a-51978c-openshift-default-757789f697-8q844\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") " pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:10.852118 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.851998    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-istio-data\") pod \"e2e-api-a-51978c-openshift-default-757789f697-8q844\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") " pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:10.852287 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.852204    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-workload-certs\") pod \"e2e-api-a-51978c-openshift-default-757789f697-8q844\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") " pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:10.852346 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.852293    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-workload-socket\") pod \"e2e-api-a-51978c-openshift-default-757789f697-8q844\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") " pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:10.852437 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.852417    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4297409-0821-412c-a8da-ac4f328b34e0-istiod-ca-cert\") pod \"e2e-api-a-51978c-openshift-default-757789f697-8q844\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") " pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:10.854156 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.854135    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-istio-envoy\") pod \"e2e-api-a-51978c-openshift-default-757789f697-8q844\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") " pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:10.854290 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.854273    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4297409-0821-412c-a8da-ac4f328b34e0-istio-podinfo\") pod \"e2e-api-a-51978c-openshift-default-757789f697-8q844\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") " pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:10.861000 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.860977    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5sdf\" (UniqueName: \"kubernetes.io/projected/e4297409-0821-412c-a8da-ac4f328b34e0-kube-api-access-p5sdf\") pod \"e2e-api-a-51978c-openshift-default-757789f697-8q844\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") " pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:10.861115 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.861033    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4297409-0821-412c-a8da-ac4f328b34e0-istio-token\") pod \"e2e-api-a-51978c-openshift-default-757789f697-8q844\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") " pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:10.966779 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:10.966695    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:11.114036 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:11.114007    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"]
Jul 07 15:34:11.116786 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:34:11.116752    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4297409_0821_412c_a8da_ac4f328b34e0.slice/crio-b5df506846b45167a0ae8d9cf4033fe33bb3f74b54ea190098b8e77c173b8ddb WatchSource:0}: Error finding container b5df506846b45167a0ae8d9cf4033fe33bb3f74b54ea190098b8e77c173b8ddb: Status 404 returned error can't find the container with id b5df506846b45167a0ae8d9cf4033fe33bb3f74b54ea190098b8e77c173b8ddb
Jul 07 15:34:11.119172 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:11.119124    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:34:11.119285 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:11.119200    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:34:11.119285 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:11.119257    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:34:11.356094 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:11.356058    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844" event={"ID":"e4297409-0821-412c-a8da-ac4f328b34e0","Type":"ContainerStarted","Data":"20689b7f6ef92b6d82d1431aae395989ba9d84a7b548abe300d7bdbaadf39963"}
Jul 07 15:34:11.356310 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:11.356113    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844" event={"ID":"e4297409-0821-412c-a8da-ac4f328b34e0","Type":"ContainerStarted","Data":"b5df506846b45167a0ae8d9cf4033fe33bb3f74b54ea190098b8e77c173b8ddb"}
Jul 07 15:34:11.380708 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:11.380652    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844" podStartSLOduration=1.380635259 podStartE2EDuration="1.380635259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:34:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:34:11.379140942 +0000 UTC m=+2019.175677672" watchObservedRunningTime="2026-07-07 15:34:11.380635259 +0000 UTC m=+2019.177171986"
Jul 07 15:34:11.967201 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:11.967164    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:12.972461 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:12.972430    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:13.367274 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:13.367226    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:13.368229 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:13.368202    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:13.485773 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:13.485731    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn"]
Jul 07 15:34:13.486550 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:13.486524    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cd769be-ffd8-423f-b8af-c61ad201f61c" containerName="istio-proxy"
Jul 07 15:34:13.490284 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:13.490261    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn"
Jul 07 15:34:13.493392 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:13.493369    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-51978c-serving-cert\""
Jul 07 15:34:13.501333 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:13.500991    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn"]
Jul 07 15:34:13.576412 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:13.576381    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6c76b69-d260-4d77-9592-22f1afb7e5b2-maas-api-tls\") pod \"maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn\" (UID: \"b6c76b69-d260-4d77-9592-22f1afb7e5b2\") " pod="opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn"
Jul 07 15:34:13.576598 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:13.576415    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6c76b69-d260-4d77-9592-22f1afb7e5b2-openshift-service-ca\") pod \"maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn\" (UID: \"b6c76b69-d260-4d77-9592-22f1afb7e5b2\") " pod="opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn"
Jul 07 15:34:13.576598 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:13.576500    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxkd6\" (UniqueName: \"kubernetes.io/projected/b6c76b69-d260-4d77-9592-22f1afb7e5b2-kube-api-access-jxkd6\") pod \"maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn\" (UID: \"b6c76b69-d260-4d77-9592-22f1afb7e5b2\") " pod="opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn"
Jul 07 15:34:13.677749 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:13.677628    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6c76b69-d260-4d77-9592-22f1afb7e5b2-maas-api-tls\") pod \"maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn\" (UID: \"b6c76b69-d260-4d77-9592-22f1afb7e5b2\") " pod="opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn"
Jul 07 15:34:13.677749 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:13.677695    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6c76b69-d260-4d77-9592-22f1afb7e5b2-openshift-service-ca\") pod \"maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn\" (UID: \"b6c76b69-d260-4d77-9592-22f1afb7e5b2\") " pod="opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn"
Jul 07 15:34:13.678067 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:13.677759    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxkd6\" (UniqueName: \"kubernetes.io/projected/b6c76b69-d260-4d77-9592-22f1afb7e5b2-kube-api-access-jxkd6\") pod \"maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn\" (UID: \"b6c76b69-d260-4d77-9592-22f1afb7e5b2\") " pod="opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn"
Jul 07 15:34:13.678067 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:34:13.677789    2567 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-51978c-serving-cert: secret "maas-api-e2e-api-a-51978c-serving-cert" not found
Jul 07 15:34:13.678067 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:34:13.677920    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6c76b69-d260-4d77-9592-22f1afb7e5b2-maas-api-tls podName:b6c76b69-d260-4d77-9592-22f1afb7e5b2 nodeName:}" failed. No retries permitted until 2026-07-07 15:34:14.177891652 +0000 UTC m=+2021.974428359 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b6c76b69-d260-4d77-9592-22f1afb7e5b2-maas-api-tls") pod "maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn" (UID: "b6c76b69-d260-4d77-9592-22f1afb7e5b2") : secret "maas-api-e2e-api-a-51978c-serving-cert" not found
Jul 07 15:34:13.678573 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:13.678551    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6c76b69-d260-4d77-9592-22f1afb7e5b2-openshift-service-ca\") pod \"maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn\" (UID: \"b6c76b69-d260-4d77-9592-22f1afb7e5b2\") " pod="opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn"
Jul 07 15:34:13.696281 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:13.696249    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxkd6\" (UniqueName: \"kubernetes.io/projected/b6c76b69-d260-4d77-9592-22f1afb7e5b2-kube-api-access-jxkd6\") pod \"maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn\" (UID: \"b6c76b69-d260-4d77-9592-22f1afb7e5b2\") " pod="opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn"
Jul 07 15:34:14.182605 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:14.182571    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6c76b69-d260-4d77-9592-22f1afb7e5b2-maas-api-tls\") pod \"maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn\" (UID: \"b6c76b69-d260-4d77-9592-22f1afb7e5b2\") " pod="opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn"
Jul 07 15:34:14.185251 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:14.185223    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6c76b69-d260-4d77-9592-22f1afb7e5b2-maas-api-tls\") pod \"maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn\" (UID: \"b6c76b69-d260-4d77-9592-22f1afb7e5b2\") " pod="opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn"
Jul 07 15:34:14.405148 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:14.405117    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn"
Jul 07 15:34:14.561200 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:14.561169    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn"]
Jul 07 15:34:14.563218 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:34:14.563192    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6c76b69_d260_4d77_9592_22f1afb7e5b2.slice/crio-e7c483d5c433c9b03804663461384588d8ad558470da9bd152711296815087fa WatchSource:0}: Error finding container e7c483d5c433c9b03804663461384588d8ad558470da9bd152711296815087fa: Status 404 returned error can't find the container with id e7c483d5c433c9b03804663461384588d8ad558470da9bd152711296815087fa
Jul 07 15:34:15.379408 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:15.379309    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn" event={"ID":"b6c76b69-d260-4d77-9592-22f1afb7e5b2","Type":"ContainerStarted","Data":"96a7df2e3abed9a2ed97724acfd3d182bebda2414efa0ddaed305026d6267de1"}
Jul 07 15:34:15.379408 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:15.379365    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn" event={"ID":"b6c76b69-d260-4d77-9592-22f1afb7e5b2","Type":"ContainerStarted","Data":"e7c483d5c433c9b03804663461384588d8ad558470da9bd152711296815087fa"}
Jul 07 15:34:15.379408 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:15.379406    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn"
Jul 07 15:34:15.401096 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:15.401046    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn" podStartSLOduration=1.878721485 podStartE2EDuration="2.401032091s" totalImagesPullingTime="522.310606ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:34:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:34:15.39914329 +0000 UTC m=+2023.195680037" watchObservedRunningTime="2026-07-07 15:34:15.401032091 +0000 UTC m=+2023.197568820"
Jul 07 15:34:21.389409 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:21.389372    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn"
Jul 07 15:34:28.953226 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:28.951855    2567 patch_prober.go:28] interesting pod/e2e-api-a-51978c-openshift-default-757789f697-8q844 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": read tcp 10.132.0.2:55808->10.132.0.73:15021: read: connection reset by peer" start-of-body=
Jul 07 15:34:28.953226 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:28.951922    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844" podUID="e4297409-0821-412c-a8da-ac4f328b34e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": read tcp 10.132.0.2:55808->10.132.0.73:15021: read: connection reset by peer"
Jul 07 15:34:29.468411 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:29.468371    2567 generic.go:356] "Generic (PLEG): container finished" podID="e4297409-0821-412c-a8da-ac4f328b34e0" containerID="20689b7f6ef92b6d82d1431aae395989ba9d84a7b548abe300d7bdbaadf39963" exitCode=0
Jul 07 15:34:29.468585 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:29.468446    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844" event={"ID":"e4297409-0821-412c-a8da-ac4f328b34e0","Type":"ContainerDied","Data":"20689b7f6ef92b6d82d1431aae395989ba9d84a7b548abe300d7bdbaadf39963"}
Jul 07 15:34:29.468877 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:29.468858    2567 scope.go:117] "RemoveContainer" containerID="20689b7f6ef92b6d82d1431aae395989ba9d84a7b548abe300d7bdbaadf39963"
Jul 07 15:34:29.469938 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:29.469910    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:34:29.470002 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:29.469981    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:34:29.470039 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:29.470014    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:34:30.476346 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:30.476311    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844" event={"ID":"e4297409-0821-412c-a8da-ac4f328b34e0","Type":"ContainerStarted","Data":"a226a4a2298451dadca32faee183d9116750be83cec47f95f0ce528e35f9ecc9"}
Jul 07 15:34:30.967626 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:30.967586    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:30.967950 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:30.967925    2567 patch_prober.go:28] interesting pod/e2e-api-a-51978c-openshift-default-757789f697-8q844 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Jul 07 15:34:30.968012 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:30.967985    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844" podUID="e4297409-0821-412c-a8da-ac4f328b34e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Jul 07 15:34:31.967150 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:31.967107    2567 patch_prober.go:28] interesting pod/e2e-api-a-51978c-openshift-default-757789f697-8q844 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Jul 07 15:34:31.967601 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:31.967177    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844" podUID="e4297409-0821-412c-a8da-ac4f328b34e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Jul 07 15:34:32.967207 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:32.967170    2567 patch_prober.go:28] interesting pod/e2e-api-a-51978c-openshift-default-757789f697-8q844 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Jul 07 15:34:32.967630 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:32.967234    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844" podUID="e4297409-0821-412c-a8da-ac4f328b34e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Jul 07 15:34:33.967579 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:33.967548    2567 patch_prober.go:28] interesting pod/e2e-api-a-51978c-openshift-default-757789f697-8q844 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Jul 07 15:34:33.967978 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:33.967616    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844" podUID="e4297409-0821-412c-a8da-ac4f328b34e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Jul 07 15:34:34.971113 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:34.971078    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:34.971574 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:34.971352    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:34.972201 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:34.972184    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:36.207919 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:36.207883    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn"]
Jul 07 15:34:36.208409 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:36.208081    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn" podUID="b6c76b69-d260-4d77-9592-22f1afb7e5b2" containerName="maas-api" containerID="cri-o://96a7df2e3abed9a2ed97724acfd3d182bebda2414efa0ddaed305026d6267de1" gracePeriod=30
Jul 07 15:34:36.464387 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:36.464327    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn"
Jul 07 15:34:36.493233 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:36.493193    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxkd6\" (UniqueName: \"kubernetes.io/projected/b6c76b69-d260-4d77-9592-22f1afb7e5b2-kube-api-access-jxkd6\") pod \"b6c76b69-d260-4d77-9592-22f1afb7e5b2\" (UID: \"b6c76b69-d260-4d77-9592-22f1afb7e5b2\") "
Jul 07 15:34:36.493405 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:36.493297    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6c76b69-d260-4d77-9592-22f1afb7e5b2-maas-api-tls\") pod \"b6c76b69-d260-4d77-9592-22f1afb7e5b2\" (UID: \"b6c76b69-d260-4d77-9592-22f1afb7e5b2\") "
Jul 07 15:34:36.493405 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:36.493343    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6c76b69-d260-4d77-9592-22f1afb7e5b2-openshift-service-ca\") pod \"b6c76b69-d260-4d77-9592-22f1afb7e5b2\" (UID: \"b6c76b69-d260-4d77-9592-22f1afb7e5b2\") "
Jul 07 15:34:36.493823 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:36.493795    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6c76b69-d260-4d77-9592-22f1afb7e5b2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b6c76b69-d260-4d77-9592-22f1afb7e5b2" (UID: "b6c76b69-d260-4d77-9592-22f1afb7e5b2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:34:36.496198 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:36.496171    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6c76b69-d260-4d77-9592-22f1afb7e5b2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b6c76b69-d260-4d77-9592-22f1afb7e5b2" (UID: "b6c76b69-d260-4d77-9592-22f1afb7e5b2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:34:36.496198 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:36.496183    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6c76b69-d260-4d77-9592-22f1afb7e5b2-kube-api-access-jxkd6" (OuterVolumeSpecName: "kube-api-access-jxkd6") pod "b6c76b69-d260-4d77-9592-22f1afb7e5b2" (UID: "b6c76b69-d260-4d77-9592-22f1afb7e5b2"). InnerVolumeSpecName "kube-api-access-jxkd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:34:36.509536 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:36.509507    2567 generic.go:356] "Generic (PLEG): container finished" podID="b6c76b69-d260-4d77-9592-22f1afb7e5b2" containerID="96a7df2e3abed9a2ed97724acfd3d182bebda2414efa0ddaed305026d6267de1" exitCode=0
Jul 07 15:34:36.509642 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:36.509548    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn"
Jul 07 15:34:36.509642 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:36.509591    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn" event={"ID":"b6c76b69-d260-4d77-9592-22f1afb7e5b2","Type":"ContainerDied","Data":"96a7df2e3abed9a2ed97724acfd3d182bebda2414efa0ddaed305026d6267de1"}
Jul 07 15:34:36.509740 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:36.509667    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn" event={"ID":"b6c76b69-d260-4d77-9592-22f1afb7e5b2","Type":"ContainerDied","Data":"e7c483d5c433c9b03804663461384588d8ad558470da9bd152711296815087fa"}
Jul 07 15:34:36.509740 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:36.509697    2567 scope.go:117] "RemoveContainer" containerID="96a7df2e3abed9a2ed97724acfd3d182bebda2414efa0ddaed305026d6267de1"
Jul 07 15:34:36.521715 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:36.521683    2567 scope.go:117] "RemoveContainer" containerID="96a7df2e3abed9a2ed97724acfd3d182bebda2414efa0ddaed305026d6267de1"
Jul 07 15:34:36.522088 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:34:36.522066    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96a7df2e3abed9a2ed97724acfd3d182bebda2414efa0ddaed305026d6267de1\": container with ID starting with 96a7df2e3abed9a2ed97724acfd3d182bebda2414efa0ddaed305026d6267de1 not found: ID does not exist" containerID="96a7df2e3abed9a2ed97724acfd3d182bebda2414efa0ddaed305026d6267de1"
Jul 07 15:34:36.522159 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:36.522097    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96a7df2e3abed9a2ed97724acfd3d182bebda2414efa0ddaed305026d6267de1"} err="failed to get container status \"96a7df2e3abed9a2ed97724acfd3d182bebda2414efa0ddaed305026d6267de1\": rpc error: code = NotFound desc = could not find container \"96a7df2e3abed9a2ed97724acfd3d182bebda2414efa0ddaed305026d6267de1\": container with ID starting with 96a7df2e3abed9a2ed97724acfd3d182bebda2414efa0ddaed305026d6267de1 not found: ID does not exist"
Jul 07 15:34:36.536307 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:36.536277    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn"]
Jul 07 15:34:36.540615 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:36.540589    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn"]
Jul 07 15:34:36.594290 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:36.594259    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxkd6\" (UniqueName: \"kubernetes.io/projected/b6c76b69-d260-4d77-9592-22f1afb7e5b2-kube-api-access-jxkd6\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:34:36.594290 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:36.594287    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6c76b69-d260-4d77-9592-22f1afb7e5b2-maas-api-tls\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:34:36.594526 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:36.594304    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6c76b69-d260-4d77-9592-22f1afb7e5b2-openshift-service-ca\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:34:36.779237 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:36.779153    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6c76b69-d260-4d77-9592-22f1afb7e5b2" path="/var/lib/kubelet/pods/b6c76b69-d260-4d77-9592-22f1afb7e5b2/volumes"
Jul 07 15:34:39.385385 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:39.385337    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-api-a-51978c-7bd8cd7b8b-qv9vn" podUID="b6c76b69-d260-4d77-9592-22f1afb7e5b2" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.74:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 07 15:34:42.458509 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:42.458475    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"]
Jul 07 15:34:42.459014 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:42.458733    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844" podUID="e4297409-0821-412c-a8da-ac4f328b34e0" containerName="istio-proxy" containerID="cri-o://a226a4a2298451dadca32faee183d9116750be83cec47f95f0ce528e35f9ecc9" gracePeriod=30
Jul 07 15:34:47.720555 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:47.720481    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:47.800722 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:47.800691    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-workload-certs\") pod \"e4297409-0821-412c-a8da-ac4f328b34e0\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") "
Jul 07 15:34:47.800934 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:47.800755    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-istio-data\") pod \"e4297409-0821-412c-a8da-ac4f328b34e0\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") "
Jul 07 15:34:47.800934 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:47.800780    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4297409-0821-412c-a8da-ac4f328b34e0-istio-token\") pod \"e4297409-0821-412c-a8da-ac4f328b34e0\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") "
Jul 07 15:34:47.800934 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:47.800858    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4297409-0821-412c-a8da-ac4f328b34e0-istio-podinfo\") pod \"e4297409-0821-412c-a8da-ac4f328b34e0\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") "
Jul 07 15:34:47.801108 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:47.800971    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-workload-socket\") pod \"e4297409-0821-412c-a8da-ac4f328b34e0\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") "
Jul 07 15:34:47.801108 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:47.801019    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5sdf\" (UniqueName: \"kubernetes.io/projected/e4297409-0821-412c-a8da-ac4f328b34e0-kube-api-access-p5sdf\") pod \"e4297409-0821-412c-a8da-ac4f328b34e0\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") "
Jul 07 15:34:47.801108 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:47.801049    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4297409-0821-412c-a8da-ac4f328b34e0-istiod-ca-cert\") pod \"e4297409-0821-412c-a8da-ac4f328b34e0\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") "
Jul 07 15:34:47.801108 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:47.801086    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e4297409-0821-412c-a8da-ac4f328b34e0" (UID: "e4297409-0821-412c-a8da-ac4f328b34e0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:34:47.801315 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:47.801111    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-credential-socket\") pod \"e4297409-0821-412c-a8da-ac4f328b34e0\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") "
Jul 07 15:34:47.801315 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:47.801135    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-istio-envoy\") pod \"e4297409-0821-412c-a8da-ac4f328b34e0\" (UID: \"e4297409-0821-412c-a8da-ac4f328b34e0\") "
Jul 07 15:34:47.801315 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:47.801257    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e4297409-0821-412c-a8da-ac4f328b34e0" (UID: "e4297409-0821-412c-a8da-ac4f328b34e0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:34:47.801484 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:47.801463    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4297409-0821-412c-a8da-ac4f328b34e0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e4297409-0821-412c-a8da-ac4f328b34e0" (UID: "e4297409-0821-412c-a8da-ac4f328b34e0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:34:47.801538 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:47.801487    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-workload-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:34:47.801538 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:47.801511    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-workload-certs\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:34:47.801735 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:47.801713    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-istio-data" (OuterVolumeSpecName: "istio-data") pod "e4297409-0821-412c-a8da-ac4f328b34e0" (UID: "e4297409-0821-412c-a8da-ac4f328b34e0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:34:47.802032 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:47.802001    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e4297409-0821-412c-a8da-ac4f328b34e0" (UID: "e4297409-0821-412c-a8da-ac4f328b34e0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:34:47.803282 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:47.803253    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4297409-0821-412c-a8da-ac4f328b34e0-istio-token" (OuterVolumeSpecName: "istio-token") pod "e4297409-0821-412c-a8da-ac4f328b34e0" (UID: "e4297409-0821-412c-a8da-ac4f328b34e0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:34:47.803282 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:47.803261    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e4297409-0821-412c-a8da-ac4f328b34e0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e4297409-0821-412c-a8da-ac4f328b34e0" (UID: "e4297409-0821-412c-a8da-ac4f328b34e0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 15:34:47.803436 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:47.803408    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e4297409-0821-412c-a8da-ac4f328b34e0" (UID: "e4297409-0821-412c-a8da-ac4f328b34e0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:34:47.803703 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:47.803682    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4297409-0821-412c-a8da-ac4f328b34e0-kube-api-access-p5sdf" (OuterVolumeSpecName: "kube-api-access-p5sdf") pod "e4297409-0821-412c-a8da-ac4f328b34e0" (UID: "e4297409-0821-412c-a8da-ac4f328b34e0"). InnerVolumeSpecName "kube-api-access-p5sdf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:34:47.903061 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:47.902875    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-istio-data\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:34:47.903061 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:47.902909    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4297409-0821-412c-a8da-ac4f328b34e0-istio-token\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:34:47.903061 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:47.902922    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4297409-0821-412c-a8da-ac4f328b34e0-istio-podinfo\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:34:47.903061 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:47.902937    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5sdf\" (UniqueName: \"kubernetes.io/projected/e4297409-0821-412c-a8da-ac4f328b34e0-kube-api-access-p5sdf\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:34:47.903061 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:47.902951    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4297409-0821-412c-a8da-ac4f328b34e0-istiod-ca-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:34:47.903061 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:47.902964    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-credential-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:34:47.903061 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:47.902978    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4297409-0821-412c-a8da-ac4f328b34e0-istio-envoy\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:34:48.578669 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:48.578633    2567 generic.go:356] "Generic (PLEG): container finished" podID="e4297409-0821-412c-a8da-ac4f328b34e0" containerID="a226a4a2298451dadca32faee183d9116750be83cec47f95f0ce528e35f9ecc9" exitCode=0
Jul 07 15:34:48.578867 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:48.578694    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"
Jul 07 15:34:48.578867 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:48.578719    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844" event={"ID":"e4297409-0821-412c-a8da-ac4f328b34e0","Type":"ContainerDied","Data":"a226a4a2298451dadca32faee183d9116750be83cec47f95f0ce528e35f9ecc9"}
Jul 07 15:34:48.578867 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:48.578802    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844" event={"ID":"e4297409-0821-412c-a8da-ac4f328b34e0","Type":"ContainerDied","Data":"b5df506846b45167a0ae8d9cf4033fe33bb3f74b54ea190098b8e77c173b8ddb"}
Jul 07 15:34:48.578867 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:48.578826    2567 scope.go:117] "RemoveContainer" containerID="a226a4a2298451dadca32faee183d9116750be83cec47f95f0ce528e35f9ecc9"
Jul 07 15:34:48.589018 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:48.588994    2567 scope.go:117] "RemoveContainer" containerID="20689b7f6ef92b6d82d1431aae395989ba9d84a7b548abe300d7bdbaadf39963"
Jul 07 15:34:48.597386 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:48.597362    2567 scope.go:117] "RemoveContainer" containerID="a226a4a2298451dadca32faee183d9116750be83cec47f95f0ce528e35f9ecc9"
Jul 07 15:34:48.597634 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:34:48.597616    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a226a4a2298451dadca32faee183d9116750be83cec47f95f0ce528e35f9ecc9\": container with ID starting with a226a4a2298451dadca32faee183d9116750be83cec47f95f0ce528e35f9ecc9 not found: ID does not exist" containerID="a226a4a2298451dadca32faee183d9116750be83cec47f95f0ce528e35f9ecc9"
Jul 07 15:34:48.597683 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:48.597644    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a226a4a2298451dadca32faee183d9116750be83cec47f95f0ce528e35f9ecc9"} err="failed to get container status \"a226a4a2298451dadca32faee183d9116750be83cec47f95f0ce528e35f9ecc9\": rpc error: code = NotFound desc = could not find container \"a226a4a2298451dadca32faee183d9116750be83cec47f95f0ce528e35f9ecc9\": container with ID starting with a226a4a2298451dadca32faee183d9116750be83cec47f95f0ce528e35f9ecc9 not found: ID does not exist"
Jul 07 15:34:48.597683 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:48.597662    2567 scope.go:117] "RemoveContainer" containerID="20689b7f6ef92b6d82d1431aae395989ba9d84a7b548abe300d7bdbaadf39963"
Jul 07 15:34:48.597960 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:34:48.597939    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20689b7f6ef92b6d82d1431aae395989ba9d84a7b548abe300d7bdbaadf39963\": container with ID starting with 20689b7f6ef92b6d82d1431aae395989ba9d84a7b548abe300d7bdbaadf39963 not found: ID does not exist" containerID="20689b7f6ef92b6d82d1431aae395989ba9d84a7b548abe300d7bdbaadf39963"
Jul 07 15:34:48.598019 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:48.597966    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20689b7f6ef92b6d82d1431aae395989ba9d84a7b548abe300d7bdbaadf39963"} err="failed to get container status \"20689b7f6ef92b6d82d1431aae395989ba9d84a7b548abe300d7bdbaadf39963\": rpc error: code = NotFound desc = could not find container \"20689b7f6ef92b6d82d1431aae395989ba9d84a7b548abe300d7bdbaadf39963\": container with ID starting with 20689b7f6ef92b6d82d1431aae395989ba9d84a7b548abe300d7bdbaadf39963 not found: ID does not exist"
Jul 07 15:34:48.604505 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:48.604469    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"]
Jul 07 15:34:48.612341 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:48.612315    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-51978c-openshift-default-757789f697-8q844"]
Jul 07 15:34:48.779141 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:48.779110    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4297409-0821-412c-a8da-ac4f328b34e0" path="/var/lib/kubelet/pods/e4297409-0821-412c-a8da-ac4f328b34e0/volumes"
Jul 07 15:34:51.038778 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.038745    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"]
Jul 07 15:34:51.039273 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.039256    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6c76b69-d260-4d77-9592-22f1afb7e5b2" containerName="maas-api"
Jul 07 15:34:51.039313 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.039275    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6c76b69-d260-4d77-9592-22f1afb7e5b2" containerName="maas-api"
Jul 07 15:34:51.039313 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.039284    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4297409-0821-412c-a8da-ac4f328b34e0" containerName="istio-proxy"
Jul 07 15:34:51.039313 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.039289    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4297409-0821-412c-a8da-ac4f328b34e0" containerName="istio-proxy"
Jul 07 15:34:51.039313 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.039295    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4297409-0821-412c-a8da-ac4f328b34e0" containerName="istio-proxy"
Jul 07 15:34:51.039313 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.039300    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4297409-0821-412c-a8da-ac4f328b34e0" containerName="istio-proxy"
Jul 07 15:34:51.039469 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.039372    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6c76b69-d260-4d77-9592-22f1afb7e5b2" containerName="maas-api"
Jul 07 15:34:51.039469 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.039382    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4297409-0821-412c-a8da-ac4f328b34e0" containerName="istio-proxy"
Jul 07 15:34:51.039469 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.039388    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4297409-0821-412c-a8da-ac4f328b34e0" containerName="istio-proxy"
Jul 07 15:34:51.044037 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.044019    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.047040 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.047018    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-8f5391-openshift-default-dockercfg-qdzsg\""
Jul 07 15:34:51.055946 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.055916    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"]
Jul 07 15:34:51.131752 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.131719    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-workload-socket\") pod \"e2e-shared-a-8f5391-openshift-default-859d657695-pzg87\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") " pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.131752 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.131758    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b672975-62d1-48f5-93b0-72c374c68fda-istio-podinfo\") pod \"e2e-shared-a-8f5391-openshift-default-859d657695-pzg87\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") " pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.132000 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.131782    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-istio-envoy\") pod \"e2e-shared-a-8f5391-openshift-default-859d657695-pzg87\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") " pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.132000 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.131913    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-credential-socket\") pod \"e2e-shared-a-8f5391-openshift-default-859d657695-pzg87\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") " pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.132000 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.131963    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-workload-certs\") pod \"e2e-shared-a-8f5391-openshift-default-859d657695-pzg87\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") " pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.132000 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.131983    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b672975-62d1-48f5-93b0-72c374c68fda-istio-token\") pod \"e2e-shared-a-8f5391-openshift-default-859d657695-pzg87\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") " pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.132135 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.132057    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b672975-62d1-48f5-93b0-72c374c68fda-istiod-ca-cert\") pod \"e2e-shared-a-8f5391-openshift-default-859d657695-pzg87\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") " pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.132135 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.132085    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-istio-data\") pod \"e2e-shared-a-8f5391-openshift-default-859d657695-pzg87\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") " pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.132135 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.132116    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr52f\" (UniqueName: \"kubernetes.io/projected/3b672975-62d1-48f5-93b0-72c374c68fda-kube-api-access-sr52f\") pod \"e2e-shared-a-8f5391-openshift-default-859d657695-pzg87\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") " pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.232810 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.232767    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-credential-socket\") pod \"e2e-shared-a-8f5391-openshift-default-859d657695-pzg87\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") " pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.233284 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.232823    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-workload-certs\") pod \"e2e-shared-a-8f5391-openshift-default-859d657695-pzg87\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") " pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.233284 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.232878    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b672975-62d1-48f5-93b0-72c374c68fda-istio-token\") pod \"e2e-shared-a-8f5391-openshift-default-859d657695-pzg87\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") " pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.233284 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.232929    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b672975-62d1-48f5-93b0-72c374c68fda-istiod-ca-cert\") pod \"e2e-shared-a-8f5391-openshift-default-859d657695-pzg87\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") " pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.233284 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.232954    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-istio-data\") pod \"e2e-shared-a-8f5391-openshift-default-859d657695-pzg87\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") " pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.233284 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.232984    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr52f\" (UniqueName: \"kubernetes.io/projected/3b672975-62d1-48f5-93b0-72c374c68fda-kube-api-access-sr52f\") pod \"e2e-shared-a-8f5391-openshift-default-859d657695-pzg87\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") " pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.233284 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.233021    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-workload-socket\") pod \"e2e-shared-a-8f5391-openshift-default-859d657695-pzg87\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") " pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.233284 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.233072    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b672975-62d1-48f5-93b0-72c374c68fda-istio-podinfo\") pod \"e2e-shared-a-8f5391-openshift-default-859d657695-pzg87\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") " pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.233284 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.233118    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-istio-envoy\") pod \"e2e-shared-a-8f5391-openshift-default-859d657695-pzg87\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") " pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.233284 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.233216    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-credential-socket\") pod \"e2e-shared-a-8f5391-openshift-default-859d657695-pzg87\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") " pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.233730 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.233328    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-istio-data\") pod \"e2e-shared-a-8f5391-openshift-default-859d657695-pzg87\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") " pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.233730 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.233379    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-workload-socket\") pod \"e2e-shared-a-8f5391-openshift-default-859d657695-pzg87\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") " pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.233730 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.233511    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-workload-certs\") pod \"e2e-shared-a-8f5391-openshift-default-859d657695-pzg87\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") " pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.233730 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.233610    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b672975-62d1-48f5-93b0-72c374c68fda-istiod-ca-cert\") pod \"e2e-shared-a-8f5391-openshift-default-859d657695-pzg87\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") " pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.235498 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.235476    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-istio-envoy\") pod \"e2e-shared-a-8f5391-openshift-default-859d657695-pzg87\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") " pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.235718 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.235699    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b672975-62d1-48f5-93b0-72c374c68fda-istio-podinfo\") pod \"e2e-shared-a-8f5391-openshift-default-859d657695-pzg87\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") " pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.242959 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.242938    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr52f\" (UniqueName: \"kubernetes.io/projected/3b672975-62d1-48f5-93b0-72c374c68fda-kube-api-access-sr52f\") pod \"e2e-shared-a-8f5391-openshift-default-859d657695-pzg87\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") " pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.244019 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.243998    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b672975-62d1-48f5-93b0-72c374c68fda-istio-token\") pod \"e2e-shared-a-8f5391-openshift-default-859d657695-pzg87\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") " pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.357052 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.357009    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:51.513906 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.513821    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"]
Jul 07 15:34:51.516292 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:34:51.516265    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b672975_62d1_48f5_93b0_72c374c68fda.slice/crio-ac73669437b5fb753cb3aa79bcf5e9eea3994de46c2d893fa4a0b853cec7c3c3 WatchSource:0}: Error finding container ac73669437b5fb753cb3aa79bcf5e9eea3994de46c2d893fa4a0b853cec7c3c3: Status 404 returned error can't find the container with id ac73669437b5fb753cb3aa79bcf5e9eea3994de46c2d893fa4a0b853cec7c3c3
Jul 07 15:34:51.518733 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.518694    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:34:51.518799 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.518780    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:34:51.518861 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.518825    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:34:51.599251 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.599199    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87" event={"ID":"3b672975-62d1-48f5-93b0-72c374c68fda","Type":"ContainerStarted","Data":"729490b57ad824c40659e84c2e9d933332d3305e17080c686a74a60f6bc46d02"}
Jul 07 15:34:51.599406 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.599269    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87" event={"ID":"3b672975-62d1-48f5-93b0-72c374c68fda","Type":"ContainerStarted","Data":"ac73669437b5fb753cb3aa79bcf5e9eea3994de46c2d893fa4a0b853cec7c3c3"}
Jul 07 15:34:51.634755 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:51.634631    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87" podStartSLOduration=0.634608508 podStartE2EDuration="634.608508ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:34:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:34:51.631703304 +0000 UTC m=+2059.428240057" watchObservedRunningTime="2026-07-07 15:34:51.634608508 +0000 UTC m=+2059.431145243"
Jul 07 15:34:52.358090 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:52.358051    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:52.363462 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:52.363435    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:52.605385 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:52.605354    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:52.606916 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:52.606889    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:34:59.014428 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.014394    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"]
Jul 07 15:34:59.019303 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.019277    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.022367 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.022338    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-600c1d-openshift-default-dockercfg-mlhbr\""
Jul 07 15:34:59.031297 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.031268    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"]
Jul 07 15:34:59.106750 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.106713    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-istio-envoy\") pod \"e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") " pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.106988 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.106774    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hskt4\" (UniqueName: \"kubernetes.io/projected/93a72317-5692-4906-a692-4a2317b180b3-kube-api-access-hskt4\") pod \"e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") " pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.106988 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.106868    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-credential-socket\") pod \"e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") " pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.106988 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.106905    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-workload-socket\") pod \"e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") " pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.106988 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.106937    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93a72317-5692-4906-a692-4a2317b180b3-istio-token\") pod \"e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") " pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.106988 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.106974    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-istio-data\") pod \"e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") " pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.107223 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.107006    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93a72317-5692-4906-a692-4a2317b180b3-istio-podinfo\") pod \"e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") " pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.107223 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.107025    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-workload-certs\") pod \"e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") " pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.107223 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.107084    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93a72317-5692-4906-a692-4a2317b180b3-istiod-ca-cert\") pod \"e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") " pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.208530 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.208494    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-workload-certs\") pod \"e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") " pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.208745 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.208555    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93a72317-5692-4906-a692-4a2317b180b3-istiod-ca-cert\") pod \"e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") " pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.208745 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.208682    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-istio-envoy\") pod \"e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") " pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.209195 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.209164    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hskt4\" (UniqueName: \"kubernetes.io/projected/93a72317-5692-4906-a692-4a2317b180b3-kube-api-access-hskt4\") pod \"e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") " pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.209439 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.209417    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-credential-socket\") pod \"e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") " pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.209570 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.209435    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-workload-certs\") pod \"e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") " pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.209666 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.209638    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-workload-socket\") pod \"e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") " pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.209743 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.209693    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-credential-socket\") pod \"e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") " pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.209743 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.209702    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93a72317-5692-4906-a692-4a2317b180b3-istio-token\") pod \"e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") " pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.209743 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.209727    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93a72317-5692-4906-a692-4a2317b180b3-istiod-ca-cert\") pod \"e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") " pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.210124 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.209777    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-istio-data\") pod \"e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") " pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.210124 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.209824    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93a72317-5692-4906-a692-4a2317b180b3-istio-podinfo\") pod \"e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") " pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.210124 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.209985    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-workload-socket\") pod \"e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") " pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.212149 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.212122    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-istio-envoy\") pod \"e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") " pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.212786 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.212760    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93a72317-5692-4906-a692-4a2317b180b3-istio-podinfo\") pod \"e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") " pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.227124 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.227071    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-istio-data\") pod \"e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") " pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.227525 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.227471    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93a72317-5692-4906-a692-4a2317b180b3-istio-token\") pod \"e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") " pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.227658 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.227566    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hskt4\" (UniqueName: \"kubernetes.io/projected/93a72317-5692-4906-a692-4a2317b180b3-kube-api-access-hskt4\") pod \"e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") " pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.334099 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.334061    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:34:59.491406 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.491377    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"]
Jul 07 15:34:59.493038 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:34:59.493003    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93a72317_5692_4906_a692_4a2317b180b3.slice/crio-eae241d60c596209c2f255625ed270c9102d4edcfb3ae248bc4f3c0ec75d9f1c WatchSource:0}: Error finding container eae241d60c596209c2f255625ed270c9102d4edcfb3ae248bc4f3c0ec75d9f1c: Status 404 returned error can't find the container with id eae241d60c596209c2f255625ed270c9102d4edcfb3ae248bc4f3c0ec75d9f1c
Jul 07 15:34:59.495245 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.495212    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:34:59.495340 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.495278    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:34:59.495340 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.495308    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:34:59.647971 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.647865    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r" event={"ID":"93a72317-5692-4906-a692-4a2317b180b3","Type":"ContainerStarted","Data":"239587f59a4abd7f3d6f30596f83b87da59a684d3436607309fc3c3d1cde0bcb"}
Jul 07 15:34:59.647971 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:34:59.647931    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r" event={"ID":"93a72317-5692-4906-a692-4a2317b180b3","Type":"ContainerStarted","Data":"eae241d60c596209c2f255625ed270c9102d4edcfb3ae248bc4f3c0ec75d9f1c"}
Jul 07 15:35:00.336290 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:00.334878    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:35:00.343636 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:00.343586    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:35:00.403462 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:00.403405    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r" podStartSLOduration=2.403390032 podStartE2EDuration="2.403390032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:34:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:34:59.673114775 +0000 UTC m=+2067.469651529" watchObservedRunningTime="2026-07-07 15:35:00.403390032 +0000 UTC m=+2068.199926761"
Jul 07 15:35:00.654104 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:00.654019    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:35:00.655079 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:00.655056    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:35:07.607333 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:07.607227    2567 patch_prober.go:28] interesting pod/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Jul 07 15:35:07.607333 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:07.607299    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87" podUID="3b672975-62d1-48f5-93b0-72c374c68fda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Jul 07 15:35:07.693520 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:07.693487    2567 generic.go:356] "Generic (PLEG): container finished" podID="3b672975-62d1-48f5-93b0-72c374c68fda" containerID="729490b57ad824c40659e84c2e9d933332d3305e17080c686a74a60f6bc46d02" exitCode=0
Jul 07 15:35:07.693728 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:07.693549    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87" event={"ID":"3b672975-62d1-48f5-93b0-72c374c68fda","Type":"ContainerDied","Data":"729490b57ad824c40659e84c2e9d933332d3305e17080c686a74a60f6bc46d02"}
Jul 07 15:35:07.694072 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:07.694039    2567 scope.go:117] "RemoveContainer" containerID="729490b57ad824c40659e84c2e9d933332d3305e17080c686a74a60f6bc46d02"
Jul 07 15:35:07.695148 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:07.695117    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:35:07.695248 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:07.695198    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:35:07.695248 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:07.695243    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:35:08.700344 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:08.700307    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87" event={"ID":"3b672975-62d1-48f5-93b0-72c374c68fda","Type":"ContainerStarted","Data":"fb7fcc1057c885f136ac938300d0bd7aad63111f7ac4437b82cab38d0756cd5c"}
Jul 07 15:35:09.358855 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:09.358251    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:35:09.358855 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:09.358422    2567 patch_prober.go:28] interesting pod/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Jul 07 15:35:09.358855 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:09.358483    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87" podUID="3b672975-62d1-48f5-93b0-72c374c68fda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Jul 07 15:35:10.357601 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:10.357561    2567 patch_prober.go:28] interesting pod/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Jul 07 15:35:10.358678 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:10.358254    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87" podUID="3b672975-62d1-48f5-93b0-72c374c68fda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Jul 07 15:35:11.357769 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:11.357730    2567 patch_prober.go:28] interesting pod/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Jul 07 15:35:11.358196 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:11.357785    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87" podUID="3b672975-62d1-48f5-93b0-72c374c68fda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Jul 07 15:35:12.358102 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:12.358052    2567 patch_prober.go:28] interesting pod/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Jul 07 15:35:12.358509 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:12.358147    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87" podUID="3b672975-62d1-48f5-93b0-72c374c68fda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Jul 07 15:35:13.362181 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:13.362152    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:35:13.362694 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:13.362679    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:35:13.363528 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:13.363502    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:35:14.738685 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:14.738655    2567 generic.go:356] "Generic (PLEG): container finished" podID="93a72317-5692-4906-a692-4a2317b180b3" containerID="239587f59a4abd7f3d6f30596f83b87da59a684d3436607309fc3c3d1cde0bcb" exitCode=0
Jul 07 15:35:14.739111 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:14.738725    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r" event={"ID":"93a72317-5692-4906-a692-4a2317b180b3","Type":"ContainerDied","Data":"239587f59a4abd7f3d6f30596f83b87da59a684d3436607309fc3c3d1cde0bcb"}
Jul 07 15:35:14.739156 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:14.739118    2567 scope.go:117] "RemoveContainer" containerID="239587f59a4abd7f3d6f30596f83b87da59a684d3436607309fc3c3d1cde0bcb"
Jul 07 15:35:14.740074 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:14.740045    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:35:14.740185 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:14.740126    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:35:14.740185 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:14.740174    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 07 15:35:15.654678 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:15.654640    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:35:15.749411 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:15.749365    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r" event={"ID":"93a72317-5692-4906-a692-4a2317b180b3","Type":"ContainerStarted","Data":"30f635f9090bb376cc19e1e3db154e0405afce47d686fabc6f369d46cf3684d2"}
Jul 07 15:35:16.334955 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:16.334919    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:35:16.336251 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:16.336218    2567 patch_prober.go:28] interesting pod/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Jul 07 15:35:16.336401 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:16.336308    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r" podUID="93a72317-5692-4906-a692-4a2317b180b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Jul 07 15:35:17.334914 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:17.334868    2567 patch_prober.go:28] interesting pod/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Jul 07 15:35:17.335429 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:17.334943    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r" podUID="93a72317-5692-4906-a692-4a2317b180b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Jul 07 15:35:18.334931 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:18.334885    2567 patch_prober.go:28] interesting pod/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Jul 07 15:35:18.335450 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:18.334959    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r" podUID="93a72317-5692-4906-a692-4a2317b180b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Jul 07 15:35:19.334993 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:19.334942    2567 patch_prober.go:28] interesting pod/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Jul 07 15:35:19.335525 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:19.335016    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r" podUID="93a72317-5692-4906-a692-4a2317b180b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Jul 07 15:35:20.338414 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:20.338384    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:35:20.338807 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:20.338723    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:35:20.339314 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:20.339296    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:35:32.847224 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:32.847184    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5r9rp_1c51cd93-b0ea-4aa1-8678-a4c833b2d37e/console-operator/1.log"
Jul 07 15:35:32.854667 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:35:32.854641    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5r9rp_1c51cd93-b0ea-4aa1-8678-a4c833b2d37e/console-operator/1.log"
Jul 07 15:36:33.020978 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:36:33.020897    2567 scope.go:117] "RemoveContainer" containerID="f4d4e104f49992d0e38d53c5a4f680899065d187bea43c4a62e9bdcaba0865ad"
Jul 07 15:40:32.878879 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:40:32.878851    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5r9rp_1c51cd93-b0ea-4aa1-8678-a4c833b2d37e/console-operator/1.log"
Jul 07 15:40:32.893602 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:40:32.893577    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5r9rp_1c51cd93-b0ea-4aa1-8678-a4c833b2d37e/console-operator/1.log"
Jul 07 15:41:22.937798 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:22.937754    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"]
Jul 07 15:41:22.938427 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:22.937980    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87" podUID="3b672975-62d1-48f5-93b0-72c374c68fda" containerName="istio-proxy" containerID="cri-o://fb7fcc1057c885f136ac938300d0bd7aad63111f7ac4437b82cab38d0756cd5c" gracePeriod=30
Jul 07 15:41:28.198534 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.198475    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:41:28.348540 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.348507    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-credential-socket\") pod \"3b672975-62d1-48f5-93b0-72c374c68fda\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") "
Jul 07 15:41:28.348768 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.348569    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-workload-certs\") pod \"3b672975-62d1-48f5-93b0-72c374c68fda\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") "
Jul 07 15:41:28.348768 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.348625    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b672975-62d1-48f5-93b0-72c374c68fda-istio-podinfo\") pod \"3b672975-62d1-48f5-93b0-72c374c68fda\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") "
Jul 07 15:41:28.348768 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.348653    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sr52f\" (UniqueName: \"kubernetes.io/projected/3b672975-62d1-48f5-93b0-72c374c68fda-kube-api-access-sr52f\") pod \"3b672975-62d1-48f5-93b0-72c374c68fda\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") "
Jul 07 15:41:28.348768 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.348677    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b672975-62d1-48f5-93b0-72c374c68fda-istiod-ca-cert\") pod \"3b672975-62d1-48f5-93b0-72c374c68fda\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") "
Jul 07 15:41:28.348768 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.348698    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-workload-socket\") pod \"3b672975-62d1-48f5-93b0-72c374c68fda\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") "
Jul 07 15:41:28.348768 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.348746    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-istio-data\") pod \"3b672975-62d1-48f5-93b0-72c374c68fda\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") "
Jul 07 15:41:28.349138 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.348795    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b672975-62d1-48f5-93b0-72c374c68fda-istio-token\") pod \"3b672975-62d1-48f5-93b0-72c374c68fda\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") "
Jul 07 15:41:28.349138 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.348814    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-istio-envoy\") pod \"3b672975-62d1-48f5-93b0-72c374c68fda\" (UID: \"3b672975-62d1-48f5-93b0-72c374c68fda\") "
Jul 07 15:41:28.349138 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.348881    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3b672975-62d1-48f5-93b0-72c374c68fda" (UID: "3b672975-62d1-48f5-93b0-72c374c68fda"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:41:28.349138 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.348989    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3b672975-62d1-48f5-93b0-72c374c68fda" (UID: "3b672975-62d1-48f5-93b0-72c374c68fda"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:41:28.349360 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.349166    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3b672975-62d1-48f5-93b0-72c374c68fda" (UID: "3b672975-62d1-48f5-93b0-72c374c68fda"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:41:28.349360 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.349117    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-credential-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:41:28.349360 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.349211    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b672975-62d1-48f5-93b0-72c374c68fda-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3b672975-62d1-48f5-93b0-72c374c68fda" (UID: "3b672975-62d1-48f5-93b0-72c374c68fda"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:41:28.349689 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.349598    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-istio-data" (OuterVolumeSpecName: "istio-data") pod "3b672975-62d1-48f5-93b0-72c374c68fda" (UID: "3b672975-62d1-48f5-93b0-72c374c68fda"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:41:28.351201 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.351171    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b672975-62d1-48f5-93b0-72c374c68fda-istio-token" (OuterVolumeSpecName: "istio-token") pod "3b672975-62d1-48f5-93b0-72c374c68fda" (UID: "3b672975-62d1-48f5-93b0-72c374c68fda"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:41:28.351201 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.351175    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3b672975-62d1-48f5-93b0-72c374c68fda" (UID: "3b672975-62d1-48f5-93b0-72c374c68fda"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:41:28.351404 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.351389    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b672975-62d1-48f5-93b0-72c374c68fda-kube-api-access-sr52f" (OuterVolumeSpecName: "kube-api-access-sr52f") pod "3b672975-62d1-48f5-93b0-72c374c68fda" (UID: "3b672975-62d1-48f5-93b0-72c374c68fda"). InnerVolumeSpecName "kube-api-access-sr52f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:41:28.351449 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.351398    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3b672975-62d1-48f5-93b0-72c374c68fda-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3b672975-62d1-48f5-93b0-72c374c68fda" (UID: "3b672975-62d1-48f5-93b0-72c374c68fda"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 15:41:28.450646 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.450548    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sr52f\" (UniqueName: \"kubernetes.io/projected/3b672975-62d1-48f5-93b0-72c374c68fda-kube-api-access-sr52f\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:41:28.450646 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.450584    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b672975-62d1-48f5-93b0-72c374c68fda-istiod-ca-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:41:28.450646 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.450598    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-workload-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:41:28.450646 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.450610    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-istio-data\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:41:28.450646 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.450621    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b672975-62d1-48f5-93b0-72c374c68fda-istio-token\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:41:28.450646 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.450633    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-istio-envoy\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:41:28.450646 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.450645    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b672975-62d1-48f5-93b0-72c374c68fda-workload-certs\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:41:28.451031 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.450656    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b672975-62d1-48f5-93b0-72c374c68fda-istio-podinfo\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:41:28.748740 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.748636    2567 generic.go:356] "Generic (PLEG): container finished" podID="3b672975-62d1-48f5-93b0-72c374c68fda" containerID="fb7fcc1057c885f136ac938300d0bd7aad63111f7ac4437b82cab38d0756cd5c" exitCode=0
Jul 07 15:41:28.748740 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.748692    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"
Jul 07 15:41:28.748740 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.748705    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87" event={"ID":"3b672975-62d1-48f5-93b0-72c374c68fda","Type":"ContainerDied","Data":"fb7fcc1057c885f136ac938300d0bd7aad63111f7ac4437b82cab38d0756cd5c"}
Jul 07 15:41:28.749058 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.748748    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87" event={"ID":"3b672975-62d1-48f5-93b0-72c374c68fda","Type":"ContainerDied","Data":"ac73669437b5fb753cb3aa79bcf5e9eea3994de46c2d893fa4a0b853cec7c3c3"}
Jul 07 15:41:28.749058 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.748769    2567 scope.go:117] "RemoveContainer" containerID="fb7fcc1057c885f136ac938300d0bd7aad63111f7ac4437b82cab38d0756cd5c"
Jul 07 15:41:28.763042 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.760896    2567 scope.go:117] "RemoveContainer" containerID="729490b57ad824c40659e84c2e9d933332d3305e17080c686a74a60f6bc46d02"
Jul 07 15:41:28.770372 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.770348    2567 scope.go:117] "RemoveContainer" containerID="fb7fcc1057c885f136ac938300d0bd7aad63111f7ac4437b82cab38d0756cd5c"
Jul 07 15:41:28.770636 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:41:28.770617    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb7fcc1057c885f136ac938300d0bd7aad63111f7ac4437b82cab38d0756cd5c\": container with ID starting with fb7fcc1057c885f136ac938300d0bd7aad63111f7ac4437b82cab38d0756cd5c not found: ID does not exist" containerID="fb7fcc1057c885f136ac938300d0bd7aad63111f7ac4437b82cab38d0756cd5c"
Jul 07 15:41:28.770678 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.770645    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb7fcc1057c885f136ac938300d0bd7aad63111f7ac4437b82cab38d0756cd5c"} err="failed to get container status \"fb7fcc1057c885f136ac938300d0bd7aad63111f7ac4437b82cab38d0756cd5c\": rpc error: code = NotFound desc = could not find container \"fb7fcc1057c885f136ac938300d0bd7aad63111f7ac4437b82cab38d0756cd5c\": container with ID starting with fb7fcc1057c885f136ac938300d0bd7aad63111f7ac4437b82cab38d0756cd5c not found: ID does not exist"
Jul 07 15:41:28.770678 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.770666    2567 scope.go:117] "RemoveContainer" containerID="729490b57ad824c40659e84c2e9d933332d3305e17080c686a74a60f6bc46d02"
Jul 07 15:41:28.770900 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:41:28.770880    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"729490b57ad824c40659e84c2e9d933332d3305e17080c686a74a60f6bc46d02\": container with ID starting with 729490b57ad824c40659e84c2e9d933332d3305e17080c686a74a60f6bc46d02 not found: ID does not exist" containerID="729490b57ad824c40659e84c2e9d933332d3305e17080c686a74a60f6bc46d02"
Jul 07 15:41:28.770960 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.770906    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"729490b57ad824c40659e84c2e9d933332d3305e17080c686a74a60f6bc46d02"} err="failed to get container status \"729490b57ad824c40659e84c2e9d933332d3305e17080c686a74a60f6bc46d02\": rpc error: code = NotFound desc = could not find container \"729490b57ad824c40659e84c2e9d933332d3305e17080c686a74a60f6bc46d02\": container with ID starting with 729490b57ad824c40659e84c2e9d933332d3305e17080c686a74a60f6bc46d02 not found: ID does not exist"
Jul 07 15:41:28.782911 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.782889    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"]
Jul 07 15:41:28.786542 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:28.786520    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-8f5391-openshift-default-859d657695-pzg87"]
Jul 07 15:41:30.781605 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:30.781572    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b672975-62d1-48f5-93b0-72c374c68fda" path="/var/lib/kubelet/pods/3b672975-62d1-48f5-93b0-72c374c68fda/volumes"
Jul 07 15:41:30.791223 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:30.791195    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"]
Jul 07 15:41:30.791466 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:30.791430    2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r" podUID="93a72317-5692-4906-a692-4a2317b180b3" containerName="istio-proxy" containerID="cri-o://30f635f9090bb376cc19e1e3db154e0405afce47d686fabc6f369d46cf3684d2" gracePeriod=30
Jul 07 15:41:35.341942 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:35.341887    2567 patch_prober.go:28] interesting pod/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 07 15:41:35.342318 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:35.341984    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r" podUID="93a72317-5692-4906-a692-4a2317b180b3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 15:41:36.066373 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.066311    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:41:36.223037 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.222992    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93a72317-5692-4906-a692-4a2317b180b3-istio-token\") pod \"93a72317-5692-4906-a692-4a2317b180b3\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") "
Jul 07 15:41:36.223037 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.223040    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-istio-envoy\") pod \"93a72317-5692-4906-a692-4a2317b180b3\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") "
Jul 07 15:41:36.223303 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.223070    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-istio-data\") pod \"93a72317-5692-4906-a692-4a2317b180b3\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") "
Jul 07 15:41:36.223303 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.223104    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93a72317-5692-4906-a692-4a2317b180b3-istiod-ca-cert\") pod \"93a72317-5692-4906-a692-4a2317b180b3\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") "
Jul 07 15:41:36.223303 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.223144    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hskt4\" (UniqueName: \"kubernetes.io/projected/93a72317-5692-4906-a692-4a2317b180b3-kube-api-access-hskt4\") pod \"93a72317-5692-4906-a692-4a2317b180b3\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") "
Jul 07 15:41:36.223303 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.223200    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93a72317-5692-4906-a692-4a2317b180b3-istio-podinfo\") pod \"93a72317-5692-4906-a692-4a2317b180b3\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") "
Jul 07 15:41:36.223303 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.223231    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-workload-certs\") pod \"93a72317-5692-4906-a692-4a2317b180b3\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") "
Jul 07 15:41:36.223303 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.223262    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-credential-socket\") pod \"93a72317-5692-4906-a692-4a2317b180b3\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") "
Jul 07 15:41:36.223594 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.223306    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-workload-socket\") pod \"93a72317-5692-4906-a692-4a2317b180b3\" (UID: \"93a72317-5692-4906-a692-4a2317b180b3\") "
Jul 07 15:41:36.223653 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.223597    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93a72317-5692-4906-a692-4a2317b180b3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "93a72317-5692-4906-a692-4a2317b180b3" (UID: "93a72317-5692-4906-a692-4a2317b180b3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:41:36.223819 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.223795    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "93a72317-5692-4906-a692-4a2317b180b3" (UID: "93a72317-5692-4906-a692-4a2317b180b3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:41:36.224063 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.224018    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "93a72317-5692-4906-a692-4a2317b180b3" (UID: "93a72317-5692-4906-a692-4a2317b180b3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:41:36.224063 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.224039    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "93a72317-5692-4906-a692-4a2317b180b3" (UID: "93a72317-5692-4906-a692-4a2317b180b3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:41:36.224527 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.224505    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-istio-data" (OuterVolumeSpecName: "istio-data") pod "93a72317-5692-4906-a692-4a2317b180b3" (UID: "93a72317-5692-4906-a692-4a2317b180b3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:41:36.225531 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.225510    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "93a72317-5692-4906-a692-4a2317b180b3" (UID: "93a72317-5692-4906-a692-4a2317b180b3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:41:36.225629 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.225565    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93a72317-5692-4906-a692-4a2317b180b3-kube-api-access-hskt4" (OuterVolumeSpecName: "kube-api-access-hskt4") pod "93a72317-5692-4906-a692-4a2317b180b3" (UID: "93a72317-5692-4906-a692-4a2317b180b3"). InnerVolumeSpecName "kube-api-access-hskt4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:41:36.225971 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.225950    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93a72317-5692-4906-a692-4a2317b180b3-istio-token" (OuterVolumeSpecName: "istio-token") pod "93a72317-5692-4906-a692-4a2317b180b3" (UID: "93a72317-5692-4906-a692-4a2317b180b3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:41:36.226065 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.226038    2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/93a72317-5692-4906-a692-4a2317b180b3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "93a72317-5692-4906-a692-4a2317b180b3" (UID: "93a72317-5692-4906-a692-4a2317b180b3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 15:41:36.324909 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.324810    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-credential-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:41:36.324909 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.324857    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-workload-socket\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:41:36.324909 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.324868    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93a72317-5692-4906-a692-4a2317b180b3-istio-token\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:41:36.324909 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.324877    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-istio-envoy\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:41:36.324909 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.324884    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-istio-data\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:41:36.324909 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.324891    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93a72317-5692-4906-a692-4a2317b180b3-istiod-ca-cert\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:41:36.324909 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.324899    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hskt4\" (UniqueName: \"kubernetes.io/projected/93a72317-5692-4906-a692-4a2317b180b3-kube-api-access-hskt4\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:41:36.324909 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.324908    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93a72317-5692-4906-a692-4a2317b180b3-istio-podinfo\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:41:36.325346 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.324915    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93a72317-5692-4906-a692-4a2317b180b3-workload-certs\") on node \"ip-10-0-138-233.ec2.internal\" DevicePath \"\""
Jul 07 15:41:36.623974 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.623895    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-dgck8_023e88fa-bc7c-4a44-be91-311ef63c3fda/manager/0.log"
Jul 07 15:41:36.746041 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.746011    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-569b75bc5d-b4jvx_4e6a50ef-eec0-4399-bef5-daf059cb5e88/maas-api/0.log"
Jul 07 15:41:36.792861 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.792812    2567 generic.go:356] "Generic (PLEG): container finished" podID="93a72317-5692-4906-a692-4a2317b180b3" containerID="30f635f9090bb376cc19e1e3db154e0405afce47d686fabc6f369d46cf3684d2" exitCode=0
Jul 07 15:41:36.793007 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.792862    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r" event={"ID":"93a72317-5692-4906-a692-4a2317b180b3","Type":"ContainerDied","Data":"30f635f9090bb376cc19e1e3db154e0405afce47d686fabc6f369d46cf3684d2"}
Jul 07 15:41:36.793007 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.792887    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"
Jul 07 15:41:36.793007 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.792914    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r" event={"ID":"93a72317-5692-4906-a692-4a2317b180b3","Type":"ContainerDied","Data":"eae241d60c596209c2f255625ed270c9102d4edcfb3ae248bc4f3c0ec75d9f1c"}
Jul 07 15:41:36.793007 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.792936    2567 scope.go:117] "RemoveContainer" containerID="30f635f9090bb376cc19e1e3db154e0405afce47d686fabc6f369d46cf3684d2"
Jul 07 15:41:36.802912 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.802877    2567 scope.go:117] "RemoveContainer" containerID="239587f59a4abd7f3d6f30596f83b87da59a684d3436607309fc3c3d1cde0bcb"
Jul 07 15:41:36.813647 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.813624    2567 scope.go:117] "RemoveContainer" containerID="30f635f9090bb376cc19e1e3db154e0405afce47d686fabc6f369d46cf3684d2"
Jul 07 15:41:36.814440 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:41:36.814409    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30f635f9090bb376cc19e1e3db154e0405afce47d686fabc6f369d46cf3684d2\": container with ID starting with 30f635f9090bb376cc19e1e3db154e0405afce47d686fabc6f369d46cf3684d2 not found: ID does not exist" containerID="30f635f9090bb376cc19e1e3db154e0405afce47d686fabc6f369d46cf3684d2"
Jul 07 15:41:36.814535 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.814449    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30f635f9090bb376cc19e1e3db154e0405afce47d686fabc6f369d46cf3684d2"} err="failed to get container status \"30f635f9090bb376cc19e1e3db154e0405afce47d686fabc6f369d46cf3684d2\": rpc error: code = NotFound desc = could not find container \"30f635f9090bb376cc19e1e3db154e0405afce47d686fabc6f369d46cf3684d2\": container with ID starting with 30f635f9090bb376cc19e1e3db154e0405afce47d686fabc6f369d46cf3684d2 not found: ID does not exist"
Jul 07 15:41:36.814535 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.814475    2567 scope.go:117] "RemoveContainer" containerID="239587f59a4abd7f3d6f30596f83b87da59a684d3436607309fc3c3d1cde0bcb"
Jul 07 15:41:36.814849 ip-10-0-138-233 kubenswrapper[2567]: E0707 15:41:36.814808    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"239587f59a4abd7f3d6f30596f83b87da59a684d3436607309fc3c3d1cde0bcb\": container with ID starting with 239587f59a4abd7f3d6f30596f83b87da59a684d3436607309fc3c3d1cde0bcb not found: ID does not exist" containerID="239587f59a4abd7f3d6f30596f83b87da59a684d3436607309fc3c3d1cde0bcb"
Jul 07 15:41:36.814964 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.814862    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"239587f59a4abd7f3d6f30596f83b87da59a684d3436607309fc3c3d1cde0bcb"} err="failed to get container status \"239587f59a4abd7f3d6f30596f83b87da59a684d3436607309fc3c3d1cde0bcb\": rpc error: code = NotFound desc = could not find container \"239587f59a4abd7f3d6f30596f83b87da59a684d3436607309fc3c3d1cde0bcb\": container with ID starting with 239587f59a4abd7f3d6f30596f83b87da59a684d3436607309fc3c3d1cde0bcb not found: ID does not exist"
Jul 07 15:41:36.816397 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.816375    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"]
Jul 07 15:41:36.820715 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.820692    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-600c1d-openshift-default-6d7b468b55-l2w8r"]
Jul 07 15:41:36.866818 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:36.866785    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723955-lsxqp_94d3966c-ea30-4fd4-98c4-b5e153421253/cleanup/0.log"
Jul 07 15:41:37.144327 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:37.144299    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5b477bdf49-mslch_5339528d-0992-43a7-90bc-1dc2d587b56d/manager/0.log"
Jul 07 15:41:37.271988 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:37.271959    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-5f6g2_f8aeca71-6370-4687-8796-6bbe61f3bc92/server/0.log"
Jul 07 15:41:37.394779 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:37.394680    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-7r9lh_17173b3e-ac3b-4761-892d-be6bb3a484c3/manager/0.log"
Jul 07 15:41:37.772297 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:37.772222    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-845dc9964f-k4w2t_be71a3b9-203e-4c8a-bd44-9994eb9813c8/copy-manifests/0.log"
Jul 07 15:41:37.785117 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:37.785082    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-845dc9964f-k4w2t_be71a3b9-203e-4c8a-bd44-9994eb9813c8/manager/0.log"
Jul 07 15:41:37.904374 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:37.904328    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-cvwzp_bfc2459b-ebe8-4a93-8978-87a7b524b49d/postgres/0.log"
Jul 07 15:41:38.778299 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:38.778265    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93a72317-5692-4906-a692-4a2317b180b3" path="/var/lib/kubelet/pods/93a72317-5692-4906-a692-4a2317b180b3/volumes"
Jul 07 15:41:39.394883 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:39.394814    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-hj44h_05c688f6-180a-4a2c-a7c1-0b3ad743666b/manager/0.log"
Jul 07 15:41:39.520696 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:39.520651    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-t6tmh_539a21f4-9e00-4fa5-bff8-2f5388906b1b/manager/0.log"
Jul 07 15:41:39.774278 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:39.774190    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-lct58_7519d06b-9629-49d3-a92e-9d267fa5ac0d/registry-server/0.log"
Jul 07 15:41:39.906598 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:39.906570    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-7k22w_1799722a-3db0-404b-b320-d3962ea95563/manager/0.log"
Jul 07 15:41:40.926593 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:40.926548    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8697c578dd-klxhp_c2338f6c-01a4-4b70-81c8-79bb3caecad7/kube-auth-proxy/0.log"
Jul 07 15:41:41.048731 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:41.048700    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-q9trj_497c7b71-8fa8-44b0-b75c-60f11cc9bd6c/istio-proxy/1.log"
Jul 07 15:41:41.176372 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:41.176340    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-6zf6z_d8ce75f8-4730-4bac-9374-04d5076960e8/payload-pre-processing/0.log"
Jul 07 15:41:41.295348 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:41.295262    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-gt4qs_8ee89224-43b1-45b8-bb94-6f1fe3d4f949/payload-processing/0.log"
Jul 07 15:41:41.406638 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:41.406612    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58948c8cff-pclzr_44e4a5b8-14bf-482c-9297-f2809ddb36c1/router/0.log"
Jul 07 15:41:41.884717 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:41.884686    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-5h6hx_e69ed3f1-1661-494b-829a-5b91cf866103/main/0.log"
Jul 07 15:41:42.133711 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:42.133670    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrtq85_39c6c995-1620-46c6-bfe2-6c2d0e402ec0/main/0.log"
Jul 07 15:41:46.391572 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:46.391533    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hkchs/must-gather-hbmdt"]
Jul 07 15:41:46.392213 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:46.392191    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b672975-62d1-48f5-93b0-72c374c68fda" containerName="istio-proxy"
Jul 07 15:41:46.392314 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:46.392216    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b672975-62d1-48f5-93b0-72c374c68fda" containerName="istio-proxy"
Jul 07 15:41:46.392314 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:46.392238    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b672975-62d1-48f5-93b0-72c374c68fda" containerName="istio-proxy"
Jul 07 15:41:46.392314 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:46.392247    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b672975-62d1-48f5-93b0-72c374c68fda" containerName="istio-proxy"
Jul 07 15:41:46.392314 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:46.392260    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93a72317-5692-4906-a692-4a2317b180b3" containerName="istio-proxy"
Jul 07 15:41:46.392314 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:46.392269    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="93a72317-5692-4906-a692-4a2317b180b3" containerName="istio-proxy"
Jul 07 15:41:46.392314 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:46.392293    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93a72317-5692-4906-a692-4a2317b180b3" containerName="istio-proxy"
Jul 07 15:41:46.392314 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:46.392301    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="93a72317-5692-4906-a692-4a2317b180b3" containerName="istio-proxy"
Jul 07 15:41:46.392667 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:46.392405    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="93a72317-5692-4906-a692-4a2317b180b3" containerName="istio-proxy"
Jul 07 15:41:46.392667 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:46.392419    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b672975-62d1-48f5-93b0-72c374c68fda" containerName="istio-proxy"
Jul 07 15:41:46.392667 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:46.392427    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="93a72317-5692-4906-a692-4a2317b180b3" containerName="istio-proxy"
Jul 07 15:41:46.392872 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:46.392854    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b672975-62d1-48f5-93b0-72c374c68fda" containerName="istio-proxy"
Jul 07 15:41:46.396545 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:46.396522    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hkchs/must-gather-hbmdt"
Jul 07 15:41:46.399431 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:46.399407    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hkchs\"/\"openshift-service-ca.crt\""
Jul 07 15:41:46.400868 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:46.400845    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hkchs\"/\"default-dockercfg-lzg8p\""
Jul 07 15:41:46.400988 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:46.400855    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hkchs\"/\"kube-root-ca.crt\""
Jul 07 15:41:46.403525 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:46.403498    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hkchs/must-gather-hbmdt"]
Jul 07 15:41:46.411878 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:46.411851    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/934c1daa-88b8-4a03-b493-0cc207115361-must-gather-output\") pod \"must-gather-hbmdt\" (UID: \"934c1daa-88b8-4a03-b493-0cc207115361\") " pod="openshift-must-gather-hkchs/must-gather-hbmdt"
Jul 07 15:41:46.412015 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:46.411915    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fndc7\" (UniqueName: \"kubernetes.io/projected/934c1daa-88b8-4a03-b493-0cc207115361-kube-api-access-fndc7\") pod \"must-gather-hbmdt\" (UID: \"934c1daa-88b8-4a03-b493-0cc207115361\") " pod="openshift-must-gather-hkchs/must-gather-hbmdt"
Jul 07 15:41:46.513296 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:46.513258    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/934c1daa-88b8-4a03-b493-0cc207115361-must-gather-output\") pod \"must-gather-hbmdt\" (UID: \"934c1daa-88b8-4a03-b493-0cc207115361\") " pod="openshift-must-gather-hkchs/must-gather-hbmdt"
Jul 07 15:41:46.513507 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:46.513319    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fndc7\" (UniqueName: \"kubernetes.io/projected/934c1daa-88b8-4a03-b493-0cc207115361-kube-api-access-fndc7\") pod \"must-gather-hbmdt\" (UID: \"934c1daa-88b8-4a03-b493-0cc207115361\") " pod="openshift-must-gather-hkchs/must-gather-hbmdt"
Jul 07 15:41:46.513707 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:46.513683    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/934c1daa-88b8-4a03-b493-0cc207115361-must-gather-output\") pod \"must-gather-hbmdt\" (UID: \"934c1daa-88b8-4a03-b493-0cc207115361\") " pod="openshift-must-gather-hkchs/must-gather-hbmdt"
Jul 07 15:41:46.524067 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:46.524041    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fndc7\" (UniqueName: \"kubernetes.io/projected/934c1daa-88b8-4a03-b493-0cc207115361-kube-api-access-fndc7\") pod \"must-gather-hbmdt\" (UID: \"934c1daa-88b8-4a03-b493-0cc207115361\") " pod="openshift-must-gather-hkchs/must-gather-hbmdt"
Jul 07 15:41:46.706977 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:46.706861    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hkchs/must-gather-hbmdt"
Jul 07 15:41:46.864334 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:46.864308    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hkchs/must-gather-hbmdt"]
Jul 07 15:41:46.866225 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:41:46.866196    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod934c1daa_88b8_4a03_b493_0cc207115361.slice/crio-a85080b3b01b92e2c7a328ce306db73c1ae82fce3574b71bc56a239345b361c5 WatchSource:0}: Error finding container a85080b3b01b92e2c7a328ce306db73c1ae82fce3574b71bc56a239345b361c5: Status 404 returned error can't find the container with id a85080b3b01b92e2c7a328ce306db73c1ae82fce3574b71bc56a239345b361c5
Jul 07 15:41:46.868065 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:46.868046    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 15:41:47.853403 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:47.853371    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hkchs/must-gather-hbmdt" event={"ID":"934c1daa-88b8-4a03-b493-0cc207115361","Type":"ContainerStarted","Data":"a85080b3b01b92e2c7a328ce306db73c1ae82fce3574b71bc56a239345b361c5"}
Jul 07 15:41:48.862898 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:48.862865    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hkchs/must-gather-hbmdt" event={"ID":"934c1daa-88b8-4a03-b493-0cc207115361","Type":"ContainerStarted","Data":"0b9b88438141f7c262c545efdc2abb18f4053db0946e85f8d9cfa8e2bf5e4dd4"}
Jul 07 15:41:48.863291 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:48.862914    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hkchs/must-gather-hbmdt" event={"ID":"934c1daa-88b8-4a03-b493-0cc207115361","Type":"ContainerStarted","Data":"6c344be0388eb8e2d66d63dd0969095fff693da13a78a8c6a1561a2a54b28839"}
Jul 07 15:41:49.582873 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:49.582822    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zz5ql_31b4f468-dc43-4504-b8c1-3f6b91c08bc5/global-pull-secret-syncer/0.log"
Jul 07 15:41:49.675445 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:49.675413    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7tg7l_671cc2c7-4b8d-4183-8a74-16d971f3431e/konnectivity-agent/0.log"
Jul 07 15:41:49.732378 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:49.732346    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-233.ec2.internal_938243ce709c303fec6b51c38490eae8/haproxy/0.log"
Jul 07 15:41:55.232420 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:55.232371    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-hj44h_05c688f6-180a-4a2c-a7c1-0b3ad743666b/manager/0.log"
Jul 07 15:41:55.265982 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:55.265944    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-t6tmh_539a21f4-9e00-4fa5-bff8-2f5388906b1b/manager/0.log"
Jul 07 15:41:55.345939 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:55.345899    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-lct58_7519d06b-9629-49d3-a92e-9d267fa5ac0d/registry-server/0.log"
Jul 07 15:41:55.489434 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:55.489354    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-7k22w_1799722a-3db0-404b-b320-d3962ea95563/manager/0.log"
Jul 07 15:41:57.526863 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:57.526808    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-qt79g_56f88c5a-6833-4eab-b858-8c4b163fbdcb/cluster-monitoring-operator/0.log"
Jul 07 15:41:57.801514 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:57.801483    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ksrg4_13d53bdc-0638-415f-b8f3-ffb33edea025/node-exporter/0.log"
Jul 07 15:41:57.826543 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:57.826456    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ksrg4_13d53bdc-0638-415f-b8f3-ffb33edea025/kube-rbac-proxy/0.log"
Jul 07 15:41:57.850736 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:57.850704    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ksrg4_13d53bdc-0638-415f-b8f3-ffb33edea025/init-textfile/0.log"
Jul 07 15:41:59.720986 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:59.720923    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-pfl6c_636cab31-5bac-436d-92e7-49a2e798e984/networking-console-plugin/0.log"
Jul 07 15:41:59.879545 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:59.879488    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hkchs/must-gather-hbmdt" podStartSLOduration=12.850541539 podStartE2EDuration="13.879470603s" totalImagesPullingTime="1.028929064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:41:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:41:48.882227758 +0000 UTC m=+2476.678764487" watchObservedRunningTime="2026-07-07 15:41:59.879470603 +0000 UTC m=+2487.676007326"
Jul 07 15:41:59.881424 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:59.881395    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hkchs/perf-node-gather-daemonset-mbt4h"]
Jul 07 15:41:59.887183 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:59.887144    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-mbt4h"
Jul 07 15:41:59.893548 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:59.893519    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hkchs/perf-node-gather-daemonset-mbt4h"]
Jul 07 15:41:59.964443 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:59.964405    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9c4t\" (UniqueName: \"kubernetes.io/projected/243c800c-b3da-4b08-9fbe-822372591bda-kube-api-access-n9c4t\") pod \"perf-node-gather-daemonset-mbt4h\" (UID: \"243c800c-b3da-4b08-9fbe-822372591bda\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-mbt4h"
Jul 07 15:41:59.964721 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:59.964701    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/243c800c-b3da-4b08-9fbe-822372591bda-proc\") pod \"perf-node-gather-daemonset-mbt4h\" (UID: \"243c800c-b3da-4b08-9fbe-822372591bda\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-mbt4h"
Jul 07 15:41:59.965002 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:59.964984    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/243c800c-b3da-4b08-9fbe-822372591bda-podres\") pod \"perf-node-gather-daemonset-mbt4h\" (UID: \"243c800c-b3da-4b08-9fbe-822372591bda\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-mbt4h"
Jul 07 15:41:59.965147 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:59.965132    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/243c800c-b3da-4b08-9fbe-822372591bda-sys\") pod \"perf-node-gather-daemonset-mbt4h\" (UID: \"243c800c-b3da-4b08-9fbe-822372591bda\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-mbt4h"
Jul 07 15:41:59.965285 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:41:59.965269    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/243c800c-b3da-4b08-9fbe-822372591bda-lib-modules\") pod \"perf-node-gather-daemonset-mbt4h\" (UID: \"243c800c-b3da-4b08-9fbe-822372591bda\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-mbt4h"
Jul 07 15:42:00.066917 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:00.066870    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/243c800c-b3da-4b08-9fbe-822372591bda-podres\") pod \"perf-node-gather-daemonset-mbt4h\" (UID: \"243c800c-b3da-4b08-9fbe-822372591bda\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-mbt4h"
Jul 07 15:42:00.067118 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:00.066933    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/243c800c-b3da-4b08-9fbe-822372591bda-sys\") pod \"perf-node-gather-daemonset-mbt4h\" (UID: \"243c800c-b3da-4b08-9fbe-822372591bda\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-mbt4h"
Jul 07 15:42:00.067118 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:00.066985    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/243c800c-b3da-4b08-9fbe-822372591bda-lib-modules\") pod \"perf-node-gather-daemonset-mbt4h\" (UID: \"243c800c-b3da-4b08-9fbe-822372591bda\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-mbt4h"
Jul 07 15:42:00.067118 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:00.066996    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/243c800c-b3da-4b08-9fbe-822372591bda-podres\") pod \"perf-node-gather-daemonset-mbt4h\" (UID: \"243c800c-b3da-4b08-9fbe-822372591bda\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-mbt4h"
Jul 07 15:42:00.067118 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:00.067029    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9c4t\" (UniqueName: \"kubernetes.io/projected/243c800c-b3da-4b08-9fbe-822372591bda-kube-api-access-n9c4t\") pod \"perf-node-gather-daemonset-mbt4h\" (UID: \"243c800c-b3da-4b08-9fbe-822372591bda\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-mbt4h"
Jul 07 15:42:00.067118 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:00.067062    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/243c800c-b3da-4b08-9fbe-822372591bda-proc\") pod \"perf-node-gather-daemonset-mbt4h\" (UID: \"243c800c-b3da-4b08-9fbe-822372591bda\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-mbt4h"
Jul 07 15:42:00.067118 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:00.067096    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/243c800c-b3da-4b08-9fbe-822372591bda-lib-modules\") pod \"perf-node-gather-daemonset-mbt4h\" (UID: \"243c800c-b3da-4b08-9fbe-822372591bda\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-mbt4h"
Jul 07 15:42:00.067434 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:00.067174    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/243c800c-b3da-4b08-9fbe-822372591bda-proc\") pod \"perf-node-gather-daemonset-mbt4h\" (UID: \"243c800c-b3da-4b08-9fbe-822372591bda\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-mbt4h"
Jul 07 15:42:00.067612 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:00.067583    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/243c800c-b3da-4b08-9fbe-822372591bda-sys\") pod \"perf-node-gather-daemonset-mbt4h\" (UID: \"243c800c-b3da-4b08-9fbe-822372591bda\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-mbt4h"
Jul 07 15:42:00.076527 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:00.076490    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9c4t\" (UniqueName: \"kubernetes.io/projected/243c800c-b3da-4b08-9fbe-822372591bda-kube-api-access-n9c4t\") pod \"perf-node-gather-daemonset-mbt4h\" (UID: \"243c800c-b3da-4b08-9fbe-822372591bda\") " pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-mbt4h"
Jul 07 15:42:00.201780 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:00.201735    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-mbt4h"
Jul 07 15:42:00.401530 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:00.401494    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hkchs/perf-node-gather-daemonset-mbt4h"]
Jul 07 15:42:00.403755 ip-10-0-138-233 kubenswrapper[2567]: W0707 15:42:00.403697    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod243c800c_b3da_4b08_9fbe_822372591bda.slice/crio-6a3c590a53ee4395da71ed999ff5536adef16641ee7aed2505b25f418b6b32d5 WatchSource:0}: Error finding container 6a3c590a53ee4395da71ed999ff5536adef16641ee7aed2505b25f418b6b32d5: Status 404 returned error can't find the container with id 6a3c590a53ee4395da71ed999ff5536adef16641ee7aed2505b25f418b6b32d5
Jul 07 15:42:00.410965 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:00.410934    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5r9rp_1c51cd93-b0ea-4aa1-8678-a4c833b2d37e/console-operator/1.log"
Jul 07 15:42:00.416966 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:00.416928    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-5r9rp_1c51cd93-b0ea-4aa1-8678-a4c833b2d37e/console-operator/2.log"
Jul 07 15:42:00.942286 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:00.942261    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-chvmp_acef1cce-5704-4d95-9418-8e87b81831bb/download-server/0.log"
Jul 07 15:42:00.969672 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:00.969617    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-mbt4h" event={"ID":"243c800c-b3da-4b08-9fbe-822372591bda","Type":"ContainerStarted","Data":"1ba44e9d7278f51ca2365d546e2d35c06ab94ed567570e913d89cec43b387dfb"}
Jul 07 15:42:00.969894 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:00.969700    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-mbt4h" event={"ID":"243c800c-b3da-4b08-9fbe-822372591bda","Type":"ContainerStarted","Data":"6a3c590a53ee4395da71ed999ff5536adef16641ee7aed2505b25f418b6b32d5"}
Jul 07 15:42:00.970076 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:00.970045    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-mbt4h"
Jul 07 15:42:00.989267 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:00.989217    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-mbt4h" podStartSLOduration=1.989200021 podStartE2EDuration="1.989200021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:41:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:42:00.988394792 +0000 UTC m=+2488.784931523" watchObservedRunningTime="2026-07-07 15:42:00.989200021 +0000 UTC m=+2488.785736786"
Jul 07 15:42:01.527590 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:01.527545    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-ds29c_f3ad7a15-08da-4441-8d40-2931adbefc20/volume-data-source-validator/0.log"
Jul 07 15:42:02.656168 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:02.656138    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rlrhq_23819f39-1440-4b6c-90fa-6db19eb48b74/dns/0.log"
Jul 07 15:42:02.678519 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:02.678491    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rlrhq_23819f39-1440-4b6c-90fa-6db19eb48b74/kube-rbac-proxy/0.log"
Jul 07 15:42:02.794811 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:02.794771    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-d2br4_4017287a-0129-4a52-8875-045408b9a82d/dns-node-resolver/0.log"
Jul 07 15:42:03.470278 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:03.470248    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zk8hk_88f0b024-94cb-44b4-a5f4-9497ce96c3cb/node-ca/0.log"
Jul 07 15:42:04.596872 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:04.596800    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8697c578dd-klxhp_c2338f6c-01a4-4b70-81c8-79bb3caecad7/kube-auth-proxy/0.log"
Jul 07 15:42:04.629349 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:04.629314    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-q9trj_497c7b71-8fa8-44b0-b75c-60f11cc9bd6c/istio-proxy/0.log"
Jul 07 15:42:04.665589 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:04.665562    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-q9trj_497c7b71-8fa8-44b0-b75c-60f11cc9bd6c/istio-proxy/1.log"
Jul 07 15:42:04.699477 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:04.699449    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-6zf6z_d8ce75f8-4730-4bac-9374-04d5076960e8/payload-pre-processing/0.log"
Jul 07 15:42:04.759738 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:04.759692    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-gt4qs_8ee89224-43b1-45b8-bb94-6f1fe3d4f949/payload-processing/0.log"
Jul 07 15:42:04.784555 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:04.784516    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58948c8cff-pclzr_44e4a5b8-14bf-482c-9297-f2809ddb36c1/router/0.log"
Jul 07 15:42:05.550724 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:05.550691    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-w9blj_dbe1e84c-636f-4d7e-adcd-67aea6cd7a34/serve-healthcheck-canary/0.log"
Jul 07 15:42:06.054247 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:06.054205    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-d5g6n_422c8ccc-828c-4491-a88b-76cd6973e056/insights-operator/0.log"
Jul 07 15:42:06.056280 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:06.056258    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-d5g6n_422c8ccc-828c-4491-a88b-76cd6973e056/insights-operator/1.log"
Jul 07 15:42:06.078678 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:06.078647    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6m26d_c81a8d96-38ae-48b5-bd30-2711751dedef/kube-rbac-proxy/0.log"
Jul 07 15:42:06.107096 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:06.107065    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6m26d_c81a8d96-38ae-48b5-bd30-2711751dedef/exporter/0.log"
Jul 07 15:42:06.130296 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:06.130269    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6m26d_c81a8d96-38ae-48b5-bd30-2711751dedef/extractor/0.log"
Jul 07 15:42:06.991271 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:06.991243    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hkchs/perf-node-gather-daemonset-mbt4h"
Jul 07 15:42:08.883006 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:08.882974    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-dgck8_023e88fa-bc7c-4a44-be91-311ef63c3fda/manager/0.log"
Jul 07 15:42:08.928804 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:08.928734    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-569b75bc5d-b4jvx_4e6a50ef-eec0-4399-bef5-daf059cb5e88/maas-api/0.log"
Jul 07 15:42:08.953521 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:08.953489    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723955-lsxqp_94d3966c-ea30-4fd4-98c4-b5e153421253/cleanup/0.log"
Jul 07 15:42:09.121734 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:09.121642    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5b477bdf49-mslch_5339528d-0992-43a7-90bc-1dc2d587b56d/manager/0.log"
Jul 07 15:42:09.145858 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:09.145312    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-5f6g2_f8aeca71-6370-4687-8796-6bbe61f3bc92/server/0.log"
Jul 07 15:42:09.230620 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:09.230589    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-7r9lh_17173b3e-ac3b-4761-892d-be6bb3a484c3/manager/0.log"
Jul 07 15:42:09.444070 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:09.443982    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-845dc9964f-k4w2t_be71a3b9-203e-4c8a-bd44-9994eb9813c8/manager/0.log"
Jul 07 15:42:09.468902 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:09.468875    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-845dc9964f-k4w2t_be71a3b9-203e-4c8a-bd44-9994eb9813c8/copy-manifests/0.log"
Jul 07 15:42:09.503780 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:09.503748    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-cvwzp_bfc2459b-ebe8-4a93-8978-87a7b524b49d/postgres/0.log"
Jul 07 15:42:10.568945 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:10.568910    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8697c578dd-klxhp_c2338f6c-01a4-4b70-81c8-79bb3caecad7/kube-auth-proxy/0.log"
Jul 07 15:42:10.586823 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:10.586793    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-q9trj_497c7b71-8fa8-44b0-b75c-60f11cc9bd6c/istio-proxy/0.log"
Jul 07 15:42:10.623867 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:10.623820    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-q9trj_497c7b71-8fa8-44b0-b75c-60f11cc9bd6c/istio-proxy/1.log"
Jul 07 15:42:10.646588 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:10.646560    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-6zf6z_d8ce75f8-4730-4bac-9374-04d5076960e8/payload-pre-processing/0.log"
Jul 07 15:42:10.689508 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:10.689457    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-gt4qs_8ee89224-43b1-45b8-bb94-6f1fe3d4f949/payload-processing/0.log"
Jul 07 15:42:10.702916 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:10.702890    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58948c8cff-pclzr_44e4a5b8-14bf-482c-9297-f2809ddb36c1/router/0.log"
Jul 07 15:42:12.170544 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:12.170468    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8697c578dd-klxhp_c2338f6c-01a4-4b70-81c8-79bb3caecad7/kube-auth-proxy/0.log"
Jul 07 15:42:12.192966 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:12.192932    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-q9trj_497c7b71-8fa8-44b0-b75c-60f11cc9bd6c/istio-proxy/0.log"
Jul 07 15:42:12.228460 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:12.228429    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-q9trj_497c7b71-8fa8-44b0-b75c-60f11cc9bd6c/istio-proxy/1.log"
Jul 07 15:42:12.251927 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:12.251895    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-6zf6z_d8ce75f8-4730-4bac-9374-04d5076960e8/payload-pre-processing/0.log"
Jul 07 15:42:12.295137 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:12.295097    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-gt4qs_8ee89224-43b1-45b8-bb94-6f1fe3d4f949/payload-processing/0.log"
Jul 07 15:42:12.309448 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:12.309415    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58948c8cff-pclzr_44e4a5b8-14bf-482c-9297-f2809ddb36c1/router/0.log"
Jul 07 15:42:15.890758 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:15.890727    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-qzhfn_e09c67d4-41b0-46a7-81a4-b556e763a739/migrator/0.log"
Jul 07 15:42:15.916704 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:15.916666    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-qzhfn_e09c67d4-41b0-46a7-81a4-b556e763a739/graceful-termination/0.log"
Jul 07 15:42:16.290424 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:16.290334    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-tfz7j_719d2a5c-4213-4e2a-bce0-9b29e0ad02c0/kube-storage-version-migrator-operator/1.log"
Jul 07 15:42:16.291211 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:16.291188    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-tfz7j_719d2a5c-4213-4e2a-bce0-9b29e0ad02c0/kube-storage-version-migrator-operator/0.log"
Jul 07 15:42:17.427358 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:17.427330    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5r27h_c6eef884-173e-4c5b-890d-3409f42fb0a3/kube-multus-additional-cni-plugins/0.log"
Jul 07 15:42:17.451573 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:17.451528    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5r27h_c6eef884-173e-4c5b-890d-3409f42fb0a3/egress-router-binary-copy/0.log"
Jul 07 15:42:17.475274 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:17.475244    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5r27h_c6eef884-173e-4c5b-890d-3409f42fb0a3/cni-plugins/0.log"
Jul 07 15:42:17.498471 ip-10-0-138-233 kubenswrapper[2567]: I0707 15:42:17.498446    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5r27h_c6eef884-173e-4c5b-890d-3409f42fb0a3/bond-cni-plugin/0.log"