Jul 30 19:22:27.665988 ip-10-0-134-180 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 30 19:22:27.666000 ip-10-0-134-180 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 30 19:22:27.666007 ip-10-0-134-180 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 30 19:22:27.666276 ip-10-0-134-180 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 30 19:22:37.860299 ip-10-0-134-180 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 30 19:22:37.860313 ip-10-0-134-180 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot c697bee46b5e4f87a48ea48372ae1b2f --
Jul 30 19:24:31.931642 ip-10-0-134-180 systemd[1]: Starting Kubernetes Kubelet...
Jul 30 19:24:32.270165 ip-10-0-134-180 kubenswrapper[2554]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 19:24:32.270165 ip-10-0-134-180 kubenswrapper[2554]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 30 19:24:32.270165 ip-10-0-134-180 kubenswrapper[2554]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 19:24:32.270165 ip-10-0-134-180 kubenswrapper[2554]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 30 19:24:32.270165 ip-10-0-134-180 kubenswrapper[2554]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 19:24:32.271034 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.270888    2554 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 30 19:24:32.277480 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277448    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 19:24:32.277480 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277472    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 19:24:32.277480 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277476    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 19:24:32.277480 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277480    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 19:24:32.277480 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277483    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 19:24:32.277480 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277486    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 19:24:32.277480 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277489    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 19:24:32.277480 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277492    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 19:24:32.277782 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277495    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 19:24:32.277782 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277497    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 19:24:32.277782 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277500    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 19:24:32.277782 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277503    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 19:24:32.277782 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277506    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 19:24:32.277782 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277508    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 19:24:32.277782 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277511    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 19:24:32.277782 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277513    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 19:24:32.277782 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277516    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 19:24:32.277782 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277518    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 19:24:32.277782 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277521    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 19:24:32.277782 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277524    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 19:24:32.277782 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277527    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 19:24:32.277782 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277530    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 19:24:32.277782 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277533    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 19:24:32.277782 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277535    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 19:24:32.277782 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277538    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 19:24:32.277782 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277540    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 19:24:32.277782 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277543    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 19:24:32.277782 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277545    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 19:24:32.278284 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277548    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 19:24:32.278284 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277551    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 19:24:32.278284 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277553    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 19:24:32.278284 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277556    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 19:24:32.278284 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277558    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 19:24:32.278284 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277561    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 19:24:32.278284 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277563    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 19:24:32.278284 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277566    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 19:24:32.278284 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277569    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 19:24:32.278284 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277572    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 19:24:32.278284 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277575    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 19:24:32.278284 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277578    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 19:24:32.278284 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277580    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 19:24:32.278284 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277583    2554 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 19:24:32.278284 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277586    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 19:24:32.278284 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277588    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 19:24:32.278284 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277591    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 19:24:32.278284 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277593    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 19:24:32.278284 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277596    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 19:24:32.278284 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277598    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 19:24:32.278828 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277601    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 19:24:32.278828 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277604    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 19:24:32.278828 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277606    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 19:24:32.278828 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277609    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 19:24:32.278828 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277612    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 19:24:32.278828 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277615    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 19:24:32.278828 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277617    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 19:24:32.278828 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277620    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 19:24:32.278828 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277622    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 19:24:32.278828 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277625    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 19:24:32.278828 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277627    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 19:24:32.278828 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277630    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 19:24:32.278828 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277632    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 19:24:32.278828 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277635    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 19:24:32.278828 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277637    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 19:24:32.278828 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277641    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 19:24:32.278828 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277646    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 19:24:32.278828 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277648    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 19:24:32.278828 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277651    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 19:24:32.278828 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277655    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 19:24:32.279336 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277658    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 19:24:32.279336 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277661    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 19:24:32.279336 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277663    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 19:24:32.279336 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277666    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 19:24:32.279336 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277668    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 19:24:32.279336 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277672    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 19:24:32.279336 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277674    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 19:24:32.279336 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277679    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 19:24:32.279336 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277684    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 19:24:32.279336 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277687    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 19:24:32.279336 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277689    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 19:24:32.279336 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277692    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 19:24:32.279336 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277695    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 19:24:32.279336 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277697    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 19:24:32.279336 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277700    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 19:24:32.279336 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277704    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 19:24:32.279336 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277708    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 19:24:32.279336 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277711    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 19:24:32.279336 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277713    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 19:24:32.279806 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277716    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 19:24:32.279806 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277719    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 19:24:32.279806 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277722    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 19:24:32.279806 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277725    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 19:24:32.279806 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277727    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 19:24:32.279806 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277730    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 19:24:32.279806 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277733    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 19:24:32.279806 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277735    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 19:24:32.279806 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277737    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 19:24:32.279806 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277740    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 19:24:32.279806 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277743    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 19:24:32.279806 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.277746    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 19:24:32.279806 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278472    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 19:24:32.279806 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278479    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 19:24:32.279806 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278482    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 19:24:32.279806 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278484    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 19:24:32.279806 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278488    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 19:24:32.279806 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278492    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 19:24:32.279806 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278495    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 19:24:32.280282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278498    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 19:24:32.280282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278500    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 19:24:32.280282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278505    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 19:24:32.280282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278509    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 19:24:32.280282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278512    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 19:24:32.280282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278515    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 19:24:32.280282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278518    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 19:24:32.280282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278520    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 19:24:32.280282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278523    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 19:24:32.280282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278525    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 19:24:32.280282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278529    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 19:24:32.280282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278531    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 19:24:32.280282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278534    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 19:24:32.280282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278536    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 19:24:32.280282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278539    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 19:24:32.280282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278541    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 19:24:32.280282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278544    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 19:24:32.280282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278546    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 19:24:32.280282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278549    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 19:24:32.280763 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278551    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 19:24:32.280763 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278554    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 19:24:32.280763 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278557    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 19:24:32.280763 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278559    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 19:24:32.280763 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278561    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 19:24:32.280763 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278564    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 19:24:32.280763 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278567    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 19:24:32.280763 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278570    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 19:24:32.280763 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278573    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 19:24:32.280763 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278576    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 19:24:32.280763 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278578    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 19:24:32.280763 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278581    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 19:24:32.280763 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278583    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 19:24:32.280763 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278586    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 19:24:32.280763 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278588    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 19:24:32.280763 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278591    2554 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 19:24:32.280763 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278593    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 19:24:32.280763 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278596    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 19:24:32.280763 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278598    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 19:24:32.280763 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278601    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 19:24:32.281282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278603    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 19:24:32.281282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278606    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 19:24:32.281282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278608    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 19:24:32.281282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278611    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 19:24:32.281282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278623    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 19:24:32.281282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278626    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 19:24:32.281282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278630    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 19:24:32.281282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278633    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 19:24:32.281282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278636    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 19:24:32.281282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278639    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 19:24:32.281282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278641    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 19:24:32.281282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278644    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 19:24:32.281282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278646    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 19:24:32.281282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278649    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 19:24:32.281282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278651    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 19:24:32.281282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278654    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 19:24:32.281282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278657    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 19:24:32.281282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278660    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 19:24:32.281282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278662    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 19:24:32.281282 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278665    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 19:24:32.281788 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278668    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 19:24:32.281788 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278671    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 19:24:32.281788 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278673    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 19:24:32.281788 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278676    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 19:24:32.281788 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278678    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 19:24:32.281788 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278681    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 19:24:32.281788 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278683    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 19:24:32.281788 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278686    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 19:24:32.281788 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278688    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 19:24:32.281788 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278691    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 19:24:32.281788 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278693    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 19:24:32.281788 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278696    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 19:24:32.281788 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278698    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 19:24:32.281788 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278701    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 19:24:32.281788 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278705    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 19:24:32.281788 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278707    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 19:24:32.281788 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278710    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 19:24:32.281788 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278713    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 19:24:32.281788 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278715    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 19:24:32.281788 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278718    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 19:24:32.282289 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278720    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 19:24:32.282289 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278723    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 19:24:32.282289 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278725    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 19:24:32.282289 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278728    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 19:24:32.282289 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278730    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 19:24:32.282289 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278733    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 19:24:32.282289 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278735    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 19:24:32.282289 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278739    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 19:24:32.282289 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278741    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 19:24:32.282289 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278766    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 19:24:32.282289 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278770    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 19:24:32.282289 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278773    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 19:24:32.282289 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.278776    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 19:24:32.282289 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279260    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 19:24:32.282289 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279266    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 19:24:32.282289 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279269    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 19:24:32.282289 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279272    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 19:24:32.282289 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279275    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 19:24:32.282289 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279279    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 19:24:32.282289 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279281    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 19:24:32.282795 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279284    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 19:24:32.282795 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279287    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 19:24:32.282795 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279290    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 19:24:32.282795 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279292    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 19:24:32.282795 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279295    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 19:24:32.282795 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279297    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 19:24:32.282795 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279300    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 19:24:32.282795 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279302    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 19:24:32.282795 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279305    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 19:24:32.282795 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279308    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 19:24:32.282795 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279311    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 19:24:32.282795 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279313    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 19:24:32.282795 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279316    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 19:24:32.282795 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279318    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 19:24:32.282795 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279321    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 19:24:32.282795 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279323    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 19:24:32.282795 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279326    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 19:24:32.282795 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279329    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 19:24:32.282795 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279331    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 19:24:32.283285 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279334    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 19:24:32.283285 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279337    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 19:24:32.283285 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279342    2554 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 19:24:32.283285 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279344    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 19:24:32.283285 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279347    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 19:24:32.283285 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279349    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 19:24:32.283285 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279352    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 19:24:32.283285 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279355    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 19:24:32.283285 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279357    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 19:24:32.283285 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279360    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 19:24:32.283285 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279363    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 19:24:32.283285 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279367    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 19:24:32.283285 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279370    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 19:24:32.283285 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279373    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 19:24:32.283285 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279375    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 19:24:32.283285 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279378    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 19:24:32.283285 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279382    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 19:24:32.283285 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279385    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 19:24:32.283285 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279388    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 19:24:32.283774 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279391    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 19:24:32.283774 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279394    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 19:24:32.283774 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279396    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 19:24:32.283774 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279399    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 19:24:32.283774 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279402    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 19:24:32.283774 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279405    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 19:24:32.283774 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279408    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 19:24:32.283774 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279411    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 19:24:32.283774 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279413    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 19:24:32.283774 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279417    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 19:24:32.283774 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279419    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 19:24:32.283774 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279421    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 19:24:32.283774 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279424    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 19:24:32.283774 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279427    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 19:24:32.283774 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279429    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 19:24:32.283774 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279433    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 19:24:32.283774 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279436    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 19:24:32.283774 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279439    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 19:24:32.283774 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279441    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 19:24:32.283774 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279444    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 19:24:32.284279 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279446    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 19:24:32.284279 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279448    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 19:24:32.284279 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279451    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 19:24:32.284279 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279453    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 19:24:32.284279 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279456    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 19:24:32.284279 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279458    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 19:24:32.284279 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279461    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 19:24:32.284279 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279463    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 19:24:32.284279 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279465    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 19:24:32.284279 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279468    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 19:24:32.284279 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279471    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 19:24:32.284279 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279473    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 19:24:32.284279 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279475    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 19:24:32.284279 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279478    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 19:24:32.284279 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279498    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 19:24:32.284279 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279502    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 19:24:32.284279 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279505    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 19:24:32.284279 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279509    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 19:24:32.284279 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279511    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 19:24:32.284279 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279514    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 19:24:32.284790 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279516    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 19:24:32.284790 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279519    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 19:24:32.284790 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279521    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 19:24:32.284790 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279524    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 19:24:32.284790 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279526    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 19:24:32.284790 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279530    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 19:24:32.284790 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279533    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 19:24:32.284790 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279537    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 19:24:32.284790 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279540    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 19:24:32.284790 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279542    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 19:24:32.284790 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279545    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 19:24:32.284790 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279548    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 19:24:32.284790 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279551    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 19:24:32.284790 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.279553    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 19:24:32.284790 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279629    2554 flags.go:64] FLAG: --address="0.0.0.0"
Jul 30 19:24:32.284790 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279639    2554 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 30 19:24:32.284790 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279649    2554 flags.go:64] FLAG: --anonymous-auth="true"
Jul 30 19:24:32.284790 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279656    2554 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 30 19:24:32.284790 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279662    2554 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 30 19:24:32.284790 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279667    2554 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 30 19:24:32.284790 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279674    2554 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 30 19:24:32.285325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279678    2554 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 30 19:24:32.285325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279682    2554 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 30 19:24:32.285325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279685    2554 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 30 19:24:32.285325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279688    2554 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 30 19:24:32.285325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279691    2554 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 30 19:24:32.285325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279694    2554 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 30 19:24:32.285325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279699    2554 flags.go:64] FLAG: --cgroup-root=""
Jul 30 19:24:32.285325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279702    2554 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 30 19:24:32.285325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279705    2554 flags.go:64] FLAG: --client-ca-file=""
Jul 30 19:24:32.285325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279713    2554 flags.go:64] FLAG: --cloud-provider="external"
Jul 30 19:24:32.285325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279716    2554 flags.go:64] FLAG: --cluster-dns="[]"
Jul 30 19:24:32.285325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279720    2554 flags.go:64] FLAG: --cluster-domain=""
Jul 30 19:24:32.285325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279723    2554 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 30 19:24:32.285325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279726    2554 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 30 19:24:32.285325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279729    2554 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 30 19:24:32.285325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279732    2554 flags.go:64] FLAG: --container-log-max-files="5"
Jul 30 19:24:32.285325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279736    2554 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 30 19:24:32.285325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279740    2554 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 30 19:24:32.285325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279743    2554 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 30 19:24:32.285325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279746    2554 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 30 19:24:32.285325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279750    2554 flags.go:64] FLAG: --contention-profiling="false"
Jul 30 19:24:32.285325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279753    2554 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 30 19:24:32.285325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279756    2554 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 30 19:24:32.285325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279759    2554 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 30 19:24:32.285907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279762    2554 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 30 19:24:32.285907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279766    2554 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 30 19:24:32.285907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279769    2554 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 30 19:24:32.285907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279772    2554 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 30 19:24:32.285907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279775    2554 flags.go:64] FLAG: --enable-load-reader="false"
Jul 30 19:24:32.285907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279778    2554 flags.go:64] FLAG: --enable-server="true"
Jul 30 19:24:32.285907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279781    2554 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 30 19:24:32.285907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279786    2554 flags.go:64] FLAG: --event-burst="100"
Jul 30 19:24:32.285907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279790    2554 flags.go:64] FLAG: --event-qps="50"
Jul 30 19:24:32.285907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279792    2554 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 30 19:24:32.285907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279795    2554 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 30 19:24:32.285907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279798    2554 flags.go:64] FLAG: --eviction-hard=""
Jul 30 19:24:32.285907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279802    2554 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 30 19:24:32.285907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279805    2554 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 30 19:24:32.285907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279810    2554 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 30 19:24:32.285907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279813    2554 flags.go:64] FLAG: --eviction-soft=""
Jul 30 19:24:32.285907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279816    2554 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 30 19:24:32.285907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279819    2554 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 30 19:24:32.285907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279823    2554 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 30 19:24:32.285907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279826    2554 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 30 19:24:32.285907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279829    2554 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 30 19:24:32.285907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279832    2554 flags.go:64] FLAG: --fail-swap-on="true"
Jul 30 19:24:32.285907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279835    2554 flags.go:64] FLAG: --feature-gates=""
Jul 30 19:24:32.285907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279839    2554 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 30 19:24:32.285907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279842    2554 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 30 19:24:32.286558 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279845    2554 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 30 19:24:32.286558 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279848    2554 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 30 19:24:32.286558 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279851    2554 flags.go:64] FLAG: --healthz-port="10248"
Jul 30 19:24:32.286558 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279854    2554 flags.go:64] FLAG: --help="false"
Jul 30 19:24:32.286558 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279857    2554 flags.go:64] FLAG: --hostname-override="ip-10-0-134-180.ec2.internal"
Jul 30 19:24:32.286558 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279860    2554 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 30 19:24:32.286558 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279863    2554 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 30 19:24:32.286558 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279865    2554 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 30 19:24:32.286558 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279869    2554 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 30 19:24:32.286558 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279872    2554 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 30 19:24:32.286558 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279875    2554 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 30 19:24:32.286558 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279879    2554 flags.go:64] FLAG: --image-service-endpoint=""
Jul 30 19:24:32.286558 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279882    2554 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 30 19:24:32.286558 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279884    2554 flags.go:64] FLAG: --kube-api-burst="100"
Jul 30 19:24:32.286558 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279887    2554 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 30 19:24:32.286558 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279890    2554 flags.go:64] FLAG: --kube-api-qps="50"
Jul 30 19:24:32.286558 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279893    2554 flags.go:64] FLAG: --kube-reserved=""
Jul 30 19:24:32.286558 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279896    2554 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 30 19:24:32.286558 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279899    2554 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 30 19:24:32.286558 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279902    2554 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 30 19:24:32.286558 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279905    2554 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 30 19:24:32.286558 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279910    2554 flags.go:64] FLAG: --lock-file=""
Jul 30 19:24:32.286558 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279913    2554 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 30 19:24:32.286558 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279917    2554 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 30 19:24:32.287238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279921    2554 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 30 19:24:32.287238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279926    2554 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 30 19:24:32.287238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279930    2554 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 30 19:24:32.287238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279933    2554 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 30 19:24:32.287238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279936    2554 flags.go:64] FLAG: --logging-format="text"
Jul 30 19:24:32.287238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279939    2554 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 30 19:24:32.287238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279942    2554 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 30 19:24:32.287238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279945    2554 flags.go:64] FLAG: --manifest-url=""
Jul 30 19:24:32.287238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279948    2554 flags.go:64] FLAG: --manifest-url-header=""
Jul 30 19:24:32.287238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279965    2554 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 30 19:24:32.287238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279968    2554 flags.go:64] FLAG: --max-open-files="1000000"
Jul 30 19:24:32.287238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279973    2554 flags.go:64] FLAG: --max-pods="110"
Jul 30 19:24:32.287238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279976    2554 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 30 19:24:32.287238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279979    2554 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 30 19:24:32.287238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279982    2554 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 30 19:24:32.287238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279985    2554 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 30 19:24:32.287238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279988    2554 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 30 19:24:32.287238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279991    2554 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 30 19:24:32.287238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.279994    2554 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 30 19:24:32.287238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280002    2554 flags.go:64] FLAG: --node-status-max-images="50"
Jul 30 19:24:32.287238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280005    2554 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 30 19:24:32.287238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280008    2554 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 30 19:24:32.287238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280011    2554 flags.go:64] FLAG: --pod-cidr=""
Jul 30 19:24:32.287800 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280014    2554 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8"
Jul 30 19:24:32.287800 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280020    2554 flags.go:64] FLAG: --pod-manifest-path=""
Jul 30 19:24:32.287800 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280023    2554 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 30 19:24:32.287800 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280026    2554 flags.go:64] FLAG: --pods-per-core="0"
Jul 30 19:24:32.287800 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280029    2554 flags.go:64] FLAG: --port="10250"
Jul 30 19:24:32.287800 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280032    2554 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 30 19:24:32.287800 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280035    2554 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0762bd81b8df70f7d"
Jul 30 19:24:32.287800 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280039    2554 flags.go:64] FLAG: --qos-reserved=""
Jul 30 19:24:32.287800 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280044    2554 flags.go:64] FLAG: --read-only-port="10255"
Jul 30 19:24:32.287800 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280047    2554 flags.go:64] FLAG: --register-node="true"
Jul 30 19:24:32.287800 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280050    2554 flags.go:64] FLAG: --register-with-taints=""
Jul 30 19:24:32.287800 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280054    2554 flags.go:64] FLAG: --registry-burst="10"
Jul 30 19:24:32.287800 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280061    2554 flags.go:64] FLAG: --registry-qps="5"
Jul 30 19:24:32.287800 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280064    2554 flags.go:64] FLAG: --reserved-cpus=""
Jul 30 19:24:32.287800 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280067    2554 flags.go:64] FLAG: --reserved-memory=""
Jul 30 19:24:32.287800 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280071    2554 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 30 19:24:32.287800 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280074    2554 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 30 19:24:32.287800 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280077    2554 flags.go:64] FLAG: --rotate-certificates="false"
Jul 30 19:24:32.287800 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280080    2554 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 30 19:24:32.287800 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280083    2554 flags.go:64] FLAG: --runonce="false"
Jul 30 19:24:32.287800 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280086    2554 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 30 19:24:32.287800 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280089    2554 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 30 19:24:32.287800 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280092    2554 flags.go:64] FLAG: --seccomp-default="false"
Jul 30 19:24:32.287800 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280095    2554 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 30 19:24:32.287800 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280098    2554 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 30 19:24:32.288520 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280101    2554 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 30 19:24:32.288520 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280104    2554 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 30 19:24:32.288520 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280107    2554 flags.go:64] FLAG: --storage-driver-password="root"
Jul 30 19:24:32.288520 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280110    2554 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 30 19:24:32.288520 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280113    2554 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 30 19:24:32.288520 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280116    2554 flags.go:64] FLAG: --storage-driver-user="root"
Jul 30 19:24:32.288520 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280119    2554 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 30 19:24:32.288520 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280122    2554 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 30 19:24:32.288520 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280125    2554 flags.go:64] FLAG: --system-cgroups=""
Jul 30 19:24:32.288520 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280128    2554 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 30 19:24:32.288520 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280133    2554 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 30 19:24:32.288520 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280136    2554 flags.go:64] FLAG: --tls-cert-file=""
Jul 30 19:24:32.288520 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280139    2554 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 30 19:24:32.288520 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280143    2554 flags.go:64] FLAG: --tls-min-version=""
Jul 30 19:24:32.288520 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280147    2554 flags.go:64] FLAG: --tls-private-key-file=""
Jul 30 19:24:32.288520 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280151    2554 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 30 19:24:32.288520 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280171    2554 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 30 19:24:32.288520 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280175    2554 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 30 19:24:32.288520 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280179    2554 flags.go:64] FLAG: --v="2"
Jul 30 19:24:32.288520 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280184    2554 flags.go:64] FLAG: --version="false"
Jul 30 19:24:32.288520 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280193    2554 flags.go:64] FLAG: --vmodule=""
Jul 30 19:24:32.288520 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280197    2554 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 30 19:24:32.288520 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.280201    2554 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 30 19:24:32.288520 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280316    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 19:24:32.288520 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280320    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 19:24:32.289155 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280324    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 19:24:32.289155 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280327    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 19:24:32.289155 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280330    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 19:24:32.289155 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280332    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 19:24:32.289155 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280335    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 19:24:32.289155 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280338    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 19:24:32.289155 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280341    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 19:24:32.289155 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280343    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 19:24:32.289155 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280346    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 19:24:32.289155 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280350    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 19:24:32.289155 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280354    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 19:24:32.289155 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280357    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 19:24:32.289155 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280359    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 19:24:32.289155 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280362    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 19:24:32.289155 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280365    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 19:24:32.289155 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280368    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 19:24:32.289155 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280372    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 19:24:32.289155 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280376    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 19:24:32.289155 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280379    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 19:24:32.289624 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280381    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 19:24:32.289624 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280384    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 19:24:32.289624 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280388    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 19:24:32.289624 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280393    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 19:24:32.289624 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280396    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 19:24:32.289624 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280398    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 19:24:32.289624 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280401    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 19:24:32.289624 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280404    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 19:24:32.289624 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280406    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 19:24:32.289624 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280409    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 19:24:32.289624 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280412    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 19:24:32.289624 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280415    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 19:24:32.289624 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280417    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 19:24:32.289624 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280426    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 19:24:32.289624 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280429    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 19:24:32.289624 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280432    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 19:24:32.289624 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280434    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 19:24:32.289624 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280436    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 19:24:32.289624 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280439    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 19:24:32.289624 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280442    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 19:24:32.290129 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280444    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 19:24:32.290129 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280447    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 19:24:32.290129 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280449    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 19:24:32.290129 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280452    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 19:24:32.290129 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280454    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 19:24:32.290129 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280457    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 19:24:32.290129 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280459    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 19:24:32.290129 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280461    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 19:24:32.290129 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280464    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 19:24:32.290129 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280466    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 19:24:32.290129 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280469    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 19:24:32.290129 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280472    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 19:24:32.290129 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280474    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 19:24:32.290129 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280477    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 19:24:32.290129 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280481    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 19:24:32.290129 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280487    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 19:24:32.290129 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280490    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 19:24:32.290129 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280493    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 19:24:32.290129 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280496    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 19:24:32.290600 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280499    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 19:24:32.290600 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280502    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 19:24:32.290600 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280505    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 19:24:32.290600 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280508    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 19:24:32.290600 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280510    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 19:24:32.290600 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280513    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 19:24:32.290600 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280516    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 19:24:32.290600 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280518    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 19:24:32.290600 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280521    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 19:24:32.290600 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280524    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 19:24:32.290600 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280526    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 19:24:32.290600 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280529    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 19:24:32.290600 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280532    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 19:24:32.290600 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280534    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 19:24:32.290600 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280537    2554 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 19:24:32.290600 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280539    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 19:24:32.290600 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280542    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 19:24:32.290600 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280544    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 19:24:32.290600 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280547    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 19:24:32.290600 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280549    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 19:24:32.291110 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280552    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 19:24:32.291110 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280554    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 19:24:32.291110 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280557    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 19:24:32.291110 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280559    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 19:24:32.291110 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280562    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 19:24:32.291110 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280565    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 19:24:32.291110 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280567    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 19:24:32.291110 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280571    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 19:24:32.291110 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280575    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 19:24:32.291110 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280578    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 19:24:32.291110 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280580    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 19:24:32.291110 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280583    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 19:24:32.291110 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280585    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 19:24:32.291110 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280588    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 19:24:32.291110 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280590    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 19:24:32.291110 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280593    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 19:24:32.291110 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280596    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 19:24:32.291110 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280598    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 19:24:32.291110 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.280601    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 19:24:32.291582 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.281198    2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 19:24:32.291582 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.288190    2554 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 30 19:24:32.291582 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.288210    2554 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 30 19:24:32.291582 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.288231    2554 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 30 19:24:32.291582 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.288237    2554 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 30 19:24:32.291582 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288297    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 19:24:32.291582 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288303    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 19:24:32.291582 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288306    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 19:24:32.291582 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288309    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 19:24:32.291582 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288312    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 19:24:32.291582 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288315    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 19:24:32.291582 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288318    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 19:24:32.291582 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288320    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 19:24:32.291582 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288323    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 19:24:32.291582 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288326    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 19:24:32.292035 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288329    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 19:24:32.292035 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288332    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 19:24:32.292035 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288334    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 19:24:32.292035 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288337    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 19:24:32.292035 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288340    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 19:24:32.292035 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288342    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 19:24:32.292035 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288344    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 19:24:32.292035 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288347    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 19:24:32.292035 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288350    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 19:24:32.292035 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288352    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 19:24:32.292035 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288355    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 19:24:32.292035 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288358    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 19:24:32.292035 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288360    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 19:24:32.292035 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288363    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 19:24:32.292035 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288366    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 19:24:32.292035 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288368    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 19:24:32.292035 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288371    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 19:24:32.292035 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288374    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 19:24:32.292035 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288376    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 19:24:32.292035 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288381    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 19:24:32.292539 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288384    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 19:24:32.292539 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288388    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 19:24:32.292539 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288391    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 19:24:32.292539 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288394    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 19:24:32.292539 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288397    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 19:24:32.292539 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288401    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 19:24:32.292539 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288404    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 19:24:32.292539 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288407    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 19:24:32.292539 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288410    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 19:24:32.292539 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288413    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 19:24:32.292539 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288416    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 19:24:32.292539 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288419    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 19:24:32.292539 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288422    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 19:24:32.292539 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288425    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 19:24:32.292539 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288427    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 19:24:32.292539 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288430    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 19:24:32.292539 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288433    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 19:24:32.292539 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288435    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 19:24:32.292539 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288438    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 19:24:32.293061 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288440    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 19:24:32.293061 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288443    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 19:24:32.293061 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288446    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 19:24:32.293061 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288448    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 19:24:32.293061 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288451    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 19:24:32.293061 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288453    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 19:24:32.293061 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288456    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 19:24:32.293061 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288458    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 19:24:32.293061 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288462    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 19:24:32.293061 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288465    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 19:24:32.293061 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288468    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 19:24:32.293061 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288473    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 19:24:32.293061 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288476    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 19:24:32.293061 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288479    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 19:24:32.293061 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288482    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 19:24:32.293061 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288484    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 19:24:32.293061 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288487    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 19:24:32.293061 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288490    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 19:24:32.293061 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288492    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 19:24:32.293538 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288495    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 19:24:32.293538 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288499    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 19:24:32.293538 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288502    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 19:24:32.293538 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288505    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 19:24:32.293538 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288508    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 19:24:32.293538 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288511    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 19:24:32.293538 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288513    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 19:24:32.293538 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288516    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 19:24:32.293538 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288519    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 19:24:32.293538 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288522    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 19:24:32.293538 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288524    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 19:24:32.293538 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288527    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 19:24:32.293538 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288530    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 19:24:32.293538 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288533    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 19:24:32.293538 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288535    2554 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 19:24:32.293538 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288538    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 19:24:32.293538 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288541    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 19:24:32.293538 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288543    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 19:24:32.293538 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288546    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 19:24:32.293538 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288548    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 19:24:32.294054 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288551    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 19:24:32.294054 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288553    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 19:24:32.294054 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288556    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 19:24:32.294054 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288559    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 19:24:32.294054 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288562    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 19:24:32.294054 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288564    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 19:24:32.294054 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288567    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 19:24:32.294054 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288570    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 19:24:32.294054 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288573    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 19:24:32.294054 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288575    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 19:24:32.294054 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288578    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 19:24:32.294054 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.288583    2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 19:24:32.294054 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288702    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 19:24:32.294054 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288705    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 19:24:32.294054 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288708    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 19:24:32.294427 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288711    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 19:24:32.294427 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288714    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 19:24:32.294427 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288717    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 19:24:32.294427 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288719    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 19:24:32.294427 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288722    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 19:24:32.294427 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288725    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 19:24:32.294427 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288727    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 19:24:32.294427 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288730    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 19:24:32.294427 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288732    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 19:24:32.294427 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288735    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 19:24:32.294427 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288738    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 19:24:32.294427 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288741    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 19:24:32.294427 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288743    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 19:24:32.294427 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288746    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 19:24:32.294427 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288748    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 19:24:32.294427 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288751    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 19:24:32.294427 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288753    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 19:24:32.294427 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288756    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 19:24:32.294427 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288758    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 19:24:32.294427 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288761    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 19:24:32.294939 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288763    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 19:24:32.294939 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288767    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 19:24:32.294939 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288770    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 19:24:32.294939 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288773    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 19:24:32.294939 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288775    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 19:24:32.294939 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288778    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 19:24:32.294939 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288781    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 19:24:32.294939 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288783    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 19:24:32.294939 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288785    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 19:24:32.294939 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288788    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 19:24:32.294939 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288790    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 19:24:32.294939 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288793    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 19:24:32.294939 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288795    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 19:24:32.294939 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288798    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 19:24:32.294939 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288800    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 19:24:32.294939 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288803    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 19:24:32.294939 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288805    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 19:24:32.294939 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288808    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 19:24:32.294939 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288811    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 19:24:32.294939 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288813    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 19:24:32.295450 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288816    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 19:24:32.295450 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288819    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 19:24:32.295450 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288823    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 19:24:32.295450 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288826    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 19:24:32.295450 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288829    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 19:24:32.295450 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288832    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 19:24:32.295450 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288834    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 19:24:32.295450 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288837    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 19:24:32.295450 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288840    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 19:24:32.295450 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288844    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 19:24:32.295450 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288847    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 19:24:32.295450 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288849    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 19:24:32.295450 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288852    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 19:24:32.295450 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288856    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 19:24:32.295450 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288858    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 19:24:32.295450 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288862    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 19:24:32.295450 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288865    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 19:24:32.295450 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288868    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 19:24:32.295901 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288871    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 19:24:32.295901 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288874    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 19:24:32.295901 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288877    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 19:24:32.295901 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288879    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 19:24:32.295901 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288882    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 19:24:32.295901 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288884    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 19:24:32.295901 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288887    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 19:24:32.295901 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288889    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 19:24:32.295901 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288892    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 19:24:32.295901 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288894    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 19:24:32.295901 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288897    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 19:24:32.295901 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288899    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 19:24:32.295901 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288902    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 19:24:32.295901 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288904    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 19:24:32.295901 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288907    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 19:24:32.295901 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288909    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 19:24:32.295901 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288912    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 19:24:32.295901 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288914    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 19:24:32.295901 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288917    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 19:24:32.296543 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288919    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 19:24:32.296543 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288922    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 19:24:32.296543 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288924    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 19:24:32.296543 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288927    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 19:24:32.296543 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288930    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 19:24:32.296543 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288932    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 19:24:32.296543 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288935    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 19:24:32.296543 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288937    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 19:24:32.296543 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288940    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 19:24:32.296543 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288942    2554 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 19:24:32.296543 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288945    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 19:24:32.296543 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288947    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 19:24:32.296543 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288965    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 19:24:32.296543 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288968    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 19:24:32.296543 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288971    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 19:24:32.296543 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288973    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 19:24:32.296543 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288976    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 19:24:32.296543 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288979    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 19:24:32.296543 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:32.288982    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 19:24:32.297342 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.288987    2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 19:24:32.297342 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.289112    2554 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 30 19:24:32.297342 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.291367    2554 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 30 19:24:32.297342 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.292773    2554 server.go:1019] "Starting client certificate rotation"
Jul 30 19:24:32.297342 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.292874    2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 19:24:32.297342 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.292910    2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 19:24:32.297342 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.295191    2554 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 19:24:32.298454 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.298435    2554 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 19:24:32.312153 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.312133    2554 log.go:25] "Validated CRI v1 runtime API"
Jul 30 19:24:32.314554 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.314538    2554 log.go:25] "Validated CRI v1 image API"
Jul 30 19:24:32.315522 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.315493    2554 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 30 19:24:32.319909 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.319882    2554 fs.go:135] Filesystem UUIDs: map[14f834ee-d40d-4d63-ae7f-8e5bb997472c:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 b3760ca1-e9b1-440b-b499-51c27f509784:/dev/nvme0n1p4]
Jul 30 19:24:32.319994 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.319907    2554 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 30 19:24:32.321520 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.321499    2554 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 19:24:32.324930 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.324814    2554 manager.go:216] Machine: {Timestamp:2026-07-30 19:24:32.323553064 +0000 UTC m=+0.288134137 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099172 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2dfc5dce3e6f540ec41bfc644fd180 SystemUUID:ec2dfc5d-ce3e-6f54-0ec4-1bfc644fd180 BootID:c697bee4-6b5e-4f87-a48e-a48372ae1b2f Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a6:25:33:79:e9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a6:25:33:79:e9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:da:54:37:d1:e9:b6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 30 19:24:32.324930 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.324927    2554 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 30 19:24:32.325074 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.325034    2554 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 30 19:24:32.327798 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.327759    2554 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 30 19:24:32.327999 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.327801    2554 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-180.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 30 19:24:32.328583 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.328570    2554 topology_manager.go:138] "Creating topology manager with none policy"
Jul 30 19:24:32.328635 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.328589    2554 container_manager_linux.go:309] "Creating device plugin manager"
Jul 30 19:24:32.328635 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.328601    2554 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 19:24:32.329072 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.329060    2554 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 19:24:32.330092 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.330080    2554 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 30 19:24:32.333726 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.333711    2554 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 19:24:32.333938 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.333925    2554 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 30 19:24:32.334030 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.333995    2554 kubelet.go:486] "Attempting to sync node with API server"
Jul 30 19:24:32.334030 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.334013    2554 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 30 19:24:32.334122 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.334035    2554 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 30 19:24:32.334122 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.334060    2554 kubelet.go:398] "Adding apiserver pod source"
Jul 30 19:24:32.334122 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.334076    2554 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 30 19:24:32.334594 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.334565    2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 19:24:32.334594 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.334591    2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 19:24:32.336774 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.336753    2554 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 30 19:24:32.339276 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.339262    2554 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 30 19:24:32.339340 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.339292    2554 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 30 19:24:32.340750 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.340737    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 30 19:24:32.340786 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.340755    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 30 19:24:32.340786 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.340761    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 30 19:24:32.340786 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.340766    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 30 19:24:32.340786 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.340775    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 30 19:24:32.340786 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.340782    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 30 19:24:32.340921 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.340790    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 30 19:24:32.340921 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.340796    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 30 19:24:32.340921 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.340804    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 30 19:24:32.340921 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.340810    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 30 19:24:32.340921 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.340819    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 30 19:24:32.340921 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.340826    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 30 19:24:32.341384 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.341374    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 30 19:24:32.341413 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.341385    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 30 19:24:32.344980 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.344945    2554 server.go:1268] "Started kubelet"
Jul 30 19:24:32.345720 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.345653    2554 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 30 19:24:32.345720 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.345692    2554 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 30 19:24:32.345865 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.345745    2554 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 30 19:24:32.346358 ip-10-0-134-180 systemd[1]: Started Kubernetes Kubelet.
Jul 30 19:24:32.346876 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.346810    2554 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-180.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 30 19:24:32.347469 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.347397    2554 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 30 19:24:32.347469 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:32.347453    2554 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-180.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 30 19:24:32.347575 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:32.347453    2554 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 30 19:24:32.347737 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.347630    2554 server.go:310] "Adding debug handlers to kubelet server"
Jul 30 19:24:32.351755 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:32.351735    2554 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 30 19:24:32.353678 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:32.352809    2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-180.ec2.internal.18c727bd8d13d9fb  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-180.ec2.internal,UID:ip-10-0-134-180.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-180.ec2.internal,},FirstTimestamp:2026-07-30 19:24:32.344881659 +0000 UTC m=+0.309462731,LastTimestamp:2026-07-30 19:24:32.344881659 +0000 UTC m=+0.309462731,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-180.ec2.internal,}"
Jul 30 19:24:32.353762 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.353723    2554 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 30 19:24:32.353762 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.353728    2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 30 19:24:32.353860 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:32.353784    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-180.ec2.internal\" not found"
Jul 30 19:24:32.354376 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.354356    2554 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 30 19:24:32.354481 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.354368    2554 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 30 19:24:32.354481 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.354394    2554 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 30 19:24:32.354579 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.354486    2554 reconstruct.go:97] "Volume reconstruction finished"
Jul 30 19:24:32.354579 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.354496    2554 reconciler.go:29] "Reconciler: start to sync state"
Jul 30 19:24:32.354579 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.354505    2554 factory.go:55] Registering systemd factory
Jul 30 19:24:32.354579 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.354521    2554 factory.go:223] Registration of the systemd container factory successfully
Jul 30 19:24:32.354790 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.354775    2554 factory.go:153] Registering CRI-O factory
Jul 30 19:24:32.354839 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.354794    2554 factory.go:223] Registration of the crio container factory successfully
Jul 30 19:24:32.354839 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.354838    2554 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 30 19:24:32.354930 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.354855    2554 factory.go:103] Registering Raw factory
Jul 30 19:24:32.354930 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.354866    2554 manager.go:1196] Started watching for new ooms in manager
Jul 30 19:24:32.355507 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.355339    2554 manager.go:349] Starting recovery of all containers
Jul 30 19:24:32.363311 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.363260    2554 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 30 19:24:32.364018 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:32.363979    2554 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-180.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 30 19:24:32.364199 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:32.364174    2554 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 30 19:24:32.364556 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.364526    2554 manager.go:354] Recovery completed
Jul 30 19:24:32.366113 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:32.366082    2554 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 30 19:24:32.369124 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.369103    2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tvp7b"
Jul 30 19:24:32.369794 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.369783    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 19:24:32.371599 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.371584    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-180.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 19:24:32.371667 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.371614    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-180.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 19:24:32.371667 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.371628    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-180.ec2.internal" event="NodeHasSufficientPID"
Jul 30 19:24:32.372276 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.372262    2554 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 30 19:24:32.372276 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.372275    2554 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 30 19:24:32.372370 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.372279    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 19:24:32.372370 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.372295    2554 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 19:24:32.372924 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.372910    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-180.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 19:24:32.373032 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.372946    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-180.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 19:24:32.373032 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.372983    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-180.ec2.internal" event="NodeHasSufficientPID"
Jul 30 19:24:32.373528 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.373517    2554 policy_none.go:49] "None policy: Start"
Jul 30 19:24:32.373565 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.373539    2554 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 30 19:24:32.373565 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.373550    2554 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 30 19:24:32.374489 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.374478    2554 policy_none.go:47] "Start"
Jul 30 19:24:32.374540 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.374527    2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tvp7b"
Jul 30 19:24:32.409725 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.409563    2554 manager.go:337] "Starting Device Plugin manager"
Jul 30 19:24:32.409849 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:32.409747    2554 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 30 19:24:32.409849 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.409762    2554 server.go:87] "Starting device plugin registration server"
Jul 30 19:24:32.410077 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.410062    2554 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 30 19:24:32.410145 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.410079    2554 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 30 19:24:32.410203 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.410163    2554 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 30 19:24:32.410278 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.410265    2554 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 30 19:24:32.410278 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.410278    2554 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 30 19:24:32.410864 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:32.410832    2554 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 30 19:24:32.410933 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:32.410893    2554 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-180.ec2.internal\" not found"
Jul 30 19:24:32.480021 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.479988    2554 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 30 19:24:32.481155 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.480041    2554 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 30 19:24:32.481155 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.480073    2554 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 30 19:24:32.481155 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:32.480115    2554 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 30 19:24:32.482668 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.482649    2554 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 19:24:32.510554 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.510519    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 19:24:32.511558 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.511540    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-180.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 19:24:32.511669 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.511575    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-180.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 19:24:32.511669 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.511588    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-180.ec2.internal" event="NodeHasSufficientPID"
Jul 30 19:24:32.511669 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.511617    2554 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-180.ec2.internal"
Jul 30 19:24:32.520925 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.520907    2554 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-180.ec2.internal"
Jul 30 19:24:32.520983 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:32.520936    2554 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-180.ec2.internal\": node \"ip-10-0-134-180.ec2.internal\" not found"
Jul 30 19:24:32.547421 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:32.547388    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-180.ec2.internal\" not found"
Jul 30 19:24:32.580993 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.580938    2554 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-180.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-180.ec2.internal"]
Jul 30 19:24:32.581155 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.581049    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 19:24:32.583001 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.582953    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-180.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 19:24:32.583126 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.583014    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-180.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 19:24:32.583126 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.583030    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-180.ec2.internal" event="NodeHasSufficientPID"
Jul 30 19:24:32.584208 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.584194    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 19:24:32.584345 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.584317    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-180.ec2.internal"
Jul 30 19:24:32.584425 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.584364    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 19:24:32.584945 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.584929    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-180.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 19:24:32.585059 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.584931    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-180.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 19:24:32.585059 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.584978    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-180.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 19:24:32.585059 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.584995    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-180.ec2.internal" event="NodeHasSufficientPID"
Jul 30 19:24:32.585059 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.585003    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-180.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 19:24:32.585059 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.585020    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-180.ec2.internal" event="NodeHasSufficientPID"
Jul 30 19:24:32.586265 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.586246    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-180.ec2.internal"
Jul 30 19:24:32.586370 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.586281    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 19:24:32.587143 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.587124    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-180.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 19:24:32.587238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.587159    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-180.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 19:24:32.587238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.587174    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-180.ec2.internal" event="NodeHasSufficientPID"
Jul 30 19:24:32.604675 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:32.604652    2554 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-180.ec2.internal\" not found" node="ip-10-0-134-180.ec2.internal"
Jul 30 19:24:32.609091 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:32.609066    2554 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-180.ec2.internal\" not found" node="ip-10-0-134-180.ec2.internal"
Jul 30 19:24:32.647972 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:32.647937    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-180.ec2.internal\" not found"
Jul 30 19:24:32.656449 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.656423    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ae111a0543c1e08f3cabab9cde34d294-config\") pod \"kube-apiserver-proxy-ip-10-0-134-180.ec2.internal\" (UID: \"ae111a0543c1e08f3cabab9cde34d294\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-180.ec2.internal"
Jul 30 19:24:32.656596 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.656455    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/15c06b079e0c6b47c947023223cc6cbc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-180.ec2.internal\" (UID: \"15c06b079e0c6b47c947023223cc6cbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-180.ec2.internal"
Jul 30 19:24:32.656596 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.656477    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15c06b079e0c6b47c947023223cc6cbc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-180.ec2.internal\" (UID: \"15c06b079e0c6b47c947023223cc6cbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-180.ec2.internal"
Jul 30 19:24:32.754002 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:32.753945    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-180.ec2.internal\" not found"
Jul 30 19:24:32.757546 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.757522    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ae111a0543c1e08f3cabab9cde34d294-config\") pod \"kube-apiserver-proxy-ip-10-0-134-180.ec2.internal\" (UID: \"ae111a0543c1e08f3cabab9cde34d294\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-180.ec2.internal"
Jul 30 19:24:32.757637 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.757562    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/15c06b079e0c6b47c947023223cc6cbc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-180.ec2.internal\" (UID: \"15c06b079e0c6b47c947023223cc6cbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-180.ec2.internal"
Jul 30 19:24:32.757637 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.757593    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15c06b079e0c6b47c947023223cc6cbc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-180.ec2.internal\" (UID: \"15c06b079e0c6b47c947023223cc6cbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-180.ec2.internal"
Jul 30 19:24:32.757736 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.757662    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ae111a0543c1e08f3cabab9cde34d294-config\") pod \"kube-apiserver-proxy-ip-10-0-134-180.ec2.internal\" (UID: \"ae111a0543c1e08f3cabab9cde34d294\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-180.ec2.internal"
Jul 30 19:24:32.757736 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.757667    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15c06b079e0c6b47c947023223cc6cbc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-180.ec2.internal\" (UID: \"15c06b079e0c6b47c947023223cc6cbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-180.ec2.internal"
Jul 30 19:24:32.757798 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.757734    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/15c06b079e0c6b47c947023223cc6cbc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-180.ec2.internal\" (UID: \"15c06b079e0c6b47c947023223cc6cbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-180.ec2.internal"
Jul 30 19:24:32.854350 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:32.854262    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-180.ec2.internal\" not found"
Jul 30 19:24:32.910968 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.910907    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-180.ec2.internal"
Jul 30 19:24:32.913229 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:32.913213    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-180.ec2.internal"
Jul 30 19:24:32.954675 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:32.954640    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-180.ec2.internal\" not found"
Jul 30 19:24:33.062011 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:33.061974    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-180.ec2.internal\" not found"
Jul 30 19:24:33.162651 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:33.162560    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-180.ec2.internal\" not found"
Jul 30 19:24:33.263258 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:33.263230    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-180.ec2.internal\" not found"
Jul 30 19:24:33.286017 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.285989    2554 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 19:24:33.292433 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.292410    2554 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 30 19:24:33.292566 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.292547    2554 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 30 19:24:33.292621 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.292577    2554 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 30 19:24:33.322133 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.322099    2554 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 19:24:33.335056 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.335026    2554 apiserver.go:52] "Watching apiserver"
Jul 30 19:24:33.345273 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.345247    2554 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 30 19:24:33.345613 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.345591    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-lk9m5","openshift-multus/multus-additional-cni-plugins-5j6j7","openshift-multus/multus-pdw24","openshift-multus/network-metrics-daemon-dtpk6","openshift-network-diagnostics/network-check-target-8nv66","openshift-ovn-kubernetes/ovnkube-node-qnhsf","kube-system/global-pull-secret-syncer-cjhlf","openshift-cluster-node-tuning-operator/tuned-rzvqk","openshift-image-registry/node-ca-cdmwn","openshift-network-operator/iptables-alerter-crvgj","kube-system/konnectivity-agent-mrw8v","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8"]
Jul 30 19:24:33.347120 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.347100    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lk9m5"
Jul 30 19:24:33.350523 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.349639    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5j6j7"
Jul 30 19:24:33.350772 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.350749    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xmj44\""
Jul 30 19:24:33.350772 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.350764    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 30 19:24:33.350900 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.350764    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 30 19:24:33.351702 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.351680    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.352629 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.352611    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 30 19:24:33.352708 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.352653    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:24:33.352772 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:33.352723    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtpk6" podUID="aec35b15-c096-4e99-a7dc-77be12a9ceca"
Jul 30 19:24:33.352979 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.352960    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nrmjb\""
Jul 30 19:24:33.353077 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.353015    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 30 19:24:33.353295 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.353281    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 30 19:24:33.353822 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.353806    2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 30 19:24:33.353884 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.353850    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:24:33.353957 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.353939    2554 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-180.ec2.internal"
Jul 30 19:24:33.353957 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:33.353942    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nv66" podUID="f218f763-1cba-4443-9784-ba8b9452bb6c"
Jul 30 19:24:33.354095 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.354023    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 30 19:24:33.354095 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.353944    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 30 19:24:33.354311 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.354295    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-szk5l\""
Jul 30 19:24:33.354387 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.354370    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 30 19:24:33.355127 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.355110    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.356326 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.356308    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:24:33.356412 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:33.356375    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjhlf" podUID="6c766173-91ba-4e51-923b-51ec626a0bee"
Jul 30 19:24:33.357417 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.357399    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.358460 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.358444    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cdmwn"
Jul 30 19:24:33.358549 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.358525    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 30 19:24:33.358549 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.358543    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 30 19:24:33.358653 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.358560    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 30 19:24:33.358709 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.358558    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-lj4fp\""
Jul 30 19:24:33.358709 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.358562    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 30 19:24:33.358915 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.358901    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 30 19:24:33.359016 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.358999    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 30 19:24:33.359757 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.359737    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 30 19:24:33.359757 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.359754    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-crvgj"
Jul 30 19:24:33.359914 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.359858    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dzj4w\""
Jul 30 19:24:33.360012 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.359995    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 30 19:24:33.360871 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.360849    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mrw8v"
Jul 30 19:24:33.361181 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.361163    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 30 19:24:33.361295 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.361280    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rkn2m\""
Jul 30 19:24:33.361504 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.361481    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 30 19:24:33.361624 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.361609    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 30 19:24:33.362114 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.362097    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8"
Jul 30 19:24:33.362208 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.362107    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 30 19:24:33.362364 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.362330    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 30 19:24:33.362482 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.362469    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 30 19:24:33.362554 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.362469    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-trzh8\""
Jul 30 19:24:33.363419 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.363284    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-v7vb2\""
Jul 30 19:24:33.363419 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.363305    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 30 19:24:33.363419 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.363349    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 30 19:24:33.363419 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.363406    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-cnibin\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.363688 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.363432    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-os-release\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.363688 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.363455    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvjgf\" (UniqueName: \"kubernetes.io/projected/6db8c394-338e-4a79-a776-1750a359ffa9-kube-api-access-gvjgf\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.363688 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.363489    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.363688 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.363517    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv624\" (UniqueName: \"kubernetes.io/projected/2b738474-0882-47d5-91ee-382257a723b3-kube-api-access-jv624\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.363688 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.363541    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-etc-kubernetes\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.363688 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.363563    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aec35b15-c096-4e99-a7dc-77be12a9ceca-metrics-certs\") pod \"network-metrics-daemon-dtpk6\" (UID: \"aec35b15-c096-4e99-a7dc-77be12a9ceca\") " pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:24:33.363688 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.363588    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-run-systemd\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.363688 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.363610    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-run-openvswitch\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.363688 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.363642    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-host-run-ovn-kubernetes\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.363688 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.363674    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-host-cni-bin\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.363688 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.363692    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2b738474-0882-47d5-91ee-382257a723b3-env-overrides\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.364228 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.363708    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2b738474-0882-47d5-91ee-382257a723b3-ovnkube-script-lib\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.364228 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.363735    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5028ef75-1c24-4d52-8364-8ff8ac66f6cf-tmp-dir\") pod \"node-resolver-lk9m5\" (UID: \"5028ef75-1c24-4d52-8364-8ff8ac66f6cf\") " pod="openshift-dns/node-resolver-lk9m5"
Jul 30 19:24:33.364228 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.363785    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-host-run-k8s-cni-cncf-io\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.364228 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.363827    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-hostroot\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.364228 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.363882    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6db8c394-338e-4a79-a776-1750a359ffa9-multus-daemon-config\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.364228 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.363928    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv49g\" (UniqueName: \"kubernetes.io/projected/f218f763-1cba-4443-9784-ba8b9452bb6c-kube-api-access-mv49g\") pod \"network-check-target-8nv66\" (UID: \"f218f763-1cba-4443-9784-ba8b9452bb6c\") " pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:24:33.364228 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.363968    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2b738474-0882-47d5-91ee-382257a723b3-ovn-node-metrics-cert\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.364228 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.364002    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-host-var-lib-kubelet\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.364228 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.364019    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-multus-conf-dir\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.364228 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.364034    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7pk5\" (UniqueName: \"kubernetes.io/projected/5028ef75-1c24-4d52-8364-8ff8ac66f6cf-kube-api-access-c7pk5\") pod \"node-resolver-lk9m5\" (UID: \"5028ef75-1c24-4d52-8364-8ff8ac66f6cf\") " pod="openshift-dns/node-resolver-lk9m5"
Jul 30 19:24:33.364228 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.364061    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-multus-socket-dir-parent\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.364228 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.364097    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-host-run-multus-certs\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.364228 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.364121    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-host-kubelet\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.364228 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.364148    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-etc-openvswitch\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.364228 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.364194    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-86lv5\""
Jul 30 19:24:33.364228 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.364193    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-node-log\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.364975 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.364253    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ad1522f-8aea-4551-af0c-131bf223b514-system-cni-dir\") pod \"multus-additional-cni-plugins-5j6j7\" (UID: \"3ad1522f-8aea-4551-af0c-131bf223b514\") " pod="openshift-multus/multus-additional-cni-plugins-5j6j7"
Jul 30 19:24:33.364975 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.364286    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6db8c394-338e-4a79-a776-1750a359ffa9-cni-binary-copy\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.364975 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.364362    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-host-run-netns\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.364975 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.364499    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-run-ovn\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.364975 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.364556    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-log-socket\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.364975 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.364617    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2b738474-0882-47d5-91ee-382257a723b3-ovnkube-config\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.364975 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.364655    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5028ef75-1c24-4d52-8364-8ff8ac66f6cf-hosts-file\") pod \"node-resolver-lk9m5\" (UID: \"5028ef75-1c24-4d52-8364-8ff8ac66f6cf\") " pod="openshift-dns/node-resolver-lk9m5"
Jul 30 19:24:33.364975 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.364710    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3ad1522f-8aea-4551-af0c-131bf223b514-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5j6j7\" (UID: \"3ad1522f-8aea-4551-af0c-131bf223b514\") " pod="openshift-multus/multus-additional-cni-plugins-5j6j7"
Jul 30 19:24:33.364975 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.364719    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 30 19:24:33.365411 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.365188    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 30 19:24:33.365411 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.364758    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3ad1522f-8aea-4551-af0c-131bf223b514-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5j6j7\" (UID: \"3ad1522f-8aea-4551-af0c-131bf223b514\") " pod="openshift-multus/multus-additional-cni-plugins-5j6j7"
Jul 30 19:24:33.365411 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.365342    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-host-var-lib-cni-multus\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.365411 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.365382    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbc5h\" (UniqueName: \"kubernetes.io/projected/aec35b15-c096-4e99-a7dc-77be12a9ceca-kube-api-access-gbc5h\") pod \"network-metrics-daemon-dtpk6\" (UID: \"aec35b15-c096-4e99-a7dc-77be12a9ceca\") " pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:24:33.365596 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.365417    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-var-lib-openvswitch\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.365596 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.365448    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3ad1522f-8aea-4551-af0c-131bf223b514-os-release\") pod \"multus-additional-cni-plugins-5j6j7\" (UID: \"3ad1522f-8aea-4551-af0c-131bf223b514\") " pod="openshift-multus/multus-additional-cni-plugins-5j6j7"
Jul 30 19:24:33.365596 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.365486    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3ad1522f-8aea-4551-af0c-131bf223b514-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5j6j7\" (UID: \"3ad1522f-8aea-4551-af0c-131bf223b514\") " pod="openshift-multus/multus-additional-cni-plugins-5j6j7"
Jul 30 19:24:33.365596 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.365518    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-multus-cni-dir\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.365831 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.365678    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-host-var-lib-cni-bin\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.365831 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.365711    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-systemd-units\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.365831 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.365487    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 30 19:24:33.366017 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.365995    2554 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 19:24:33.366086 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.366073    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-host-run-netns\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.366173 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.366115    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc4q7\" (UniqueName: \"kubernetes.io/projected/3ad1522f-8aea-4551-af0c-131bf223b514-kube-api-access-jc4q7\") pod \"multus-additional-cni-plugins-5j6j7\" (UID: \"3ad1522f-8aea-4551-af0c-131bf223b514\") " pod="openshift-multus/multus-additional-cni-plugins-5j6j7"
Jul 30 19:24:33.366306 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.366229    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3ad1522f-8aea-4551-af0c-131bf223b514-cnibin\") pod \"multus-additional-cni-plugins-5j6j7\" (UID: \"3ad1522f-8aea-4551-af0c-131bf223b514\") " pod="openshift-multus/multus-additional-cni-plugins-5j6j7"
Jul 30 19:24:33.366306 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.366265    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3ad1522f-8aea-4551-af0c-131bf223b514-cni-binary-copy\") pod \"multus-additional-cni-plugins-5j6j7\" (UID: \"3ad1522f-8aea-4551-af0c-131bf223b514\") " pod="openshift-multus/multus-additional-cni-plugins-5j6j7"
Jul 30 19:24:33.366306 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.366295    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-system-cni-dir\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.366535 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.366334    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-host-slash\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.366535 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.366499    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-host-cni-netd\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.370150 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.369567    2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 19:24:33.370150 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.369791    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-180.ec2.internal"]
Jul 30 19:24:33.370397 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.370382    2554 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-180.ec2.internal"
Jul 30 19:24:33.378413 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.378372    2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-29 19:19:32 +0000 UTC" deadline="2028-04-19 03:42:17.633560592 +0000 UTC"
Jul 30 19:24:33.378489 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.378413    2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15080h17m44.255150193s"
Jul 30 19:24:33.379541 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.379522    2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 19:24:33.379628 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.379617    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-134-180.ec2.internal"]
Jul 30 19:24:33.384940 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.384919    2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nt2kw"
Jul 30 19:24:33.393131 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.393109    2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nt2kw"
Jul 30 19:24:33.455849 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.455822    2554 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 30 19:24:33.466732 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.466697    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-run-systemd\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.466914 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.466746    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-run-openvswitch\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.466914 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.466802    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-run-systemd\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.466914 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.466806    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-run-openvswitch\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.466914 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.466863    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-host-run-ovn-kubernetes\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.466914 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.466891    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2b738474-0882-47d5-91ee-382257a723b3-env-overrides\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.466914 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.466915    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2b738474-0882-47d5-91ee-382257a723b3-ovnkube-script-lib\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.467369 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.466939    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5028ef75-1c24-4d52-8364-8ff8ac66f6cf-tmp-dir\") pod \"node-resolver-lk9m5\" (UID: \"5028ef75-1c24-4d52-8364-8ff8ac66f6cf\") " pod="openshift-dns/node-resolver-lk9m5"
Jul 30 19:24:33.467369 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.466965    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-host-run-k8s-cni-cncf-io\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.467369 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.466977    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-host-run-ovn-kubernetes\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.467369 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.466990    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-hostroot\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.467369 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.467038    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv49g\" (UniqueName: \"kubernetes.io/projected/f218f763-1cba-4443-9784-ba8b9452bb6c-kube-api-access-mv49g\") pod \"network-check-target-8nv66\" (UID: \"f218f763-1cba-4443-9784-ba8b9452bb6c\") " pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:24:33.467369 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.467063    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-lib-modules\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.467369 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.467079    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a5396a9a-e676-4ef3-b9a3-48e0236d329a-etc-tuned\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.467369 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.467119    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-hostroot\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.467369 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.467154    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-host-run-k8s-cni-cncf-io\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.467369 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.467164    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/617b11a2-dd97-4d93-9078-6a1225762223-host-slash\") pod \"iptables-alerter-crvgj\" (UID: \"617b11a2-dd97-4d93-9078-6a1225762223\") " pod="openshift-network-operator/iptables-alerter-crvgj"
Jul 30 19:24:33.467369 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.467241    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d7b84b62-b1c3-4ad0-a65b-e8b1238498d1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gvfg8\" (UID: \"d7b84b62-b1c3-4ad0-a65b-e8b1238498d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8"
Jul 30 19:24:33.467985 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.467384    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-multus-conf-dir\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.467985 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.467418    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-multus-socket-dir-parent\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.467985 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.467494    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-multus-socket-dir-parent\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.467985 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.467494    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-multus-conf-dir\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.467985 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.467512    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5028ef75-1c24-4d52-8364-8ff8ac66f6cf-tmp-dir\") pod \"node-resolver-lk9m5\" (UID: \"5028ef75-1c24-4d52-8364-8ff8ac66f6cf\") " pod="openshift-dns/node-resolver-lk9m5"
Jul 30 19:24:33.467985 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.467533    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-host-run-multus-certs\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.467985 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.467568    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-host-kubelet\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.467985 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.467601    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-host-run-multus-certs\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.467985 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.467604    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-log-socket\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.467985 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.467640    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-log-socket\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.467985 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.467644    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-host-kubelet\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.467985 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.467641    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a5396a9a-e676-4ef3-b9a3-48e0236d329a-tmp\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.467985 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.467645    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2b738474-0882-47d5-91ee-382257a723b3-env-overrides\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.467985 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.467699    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52nn5\" (UniqueName: \"kubernetes.io/projected/d7b84b62-b1c3-4ad0-a65b-e8b1238498d1-kube-api-access-52nn5\") pod \"aws-ebs-csi-driver-node-gvfg8\" (UID: \"d7b84b62-b1c3-4ad0-a65b-e8b1238498d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8"
Jul 30 19:24:33.467985 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.467735    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2b738474-0882-47d5-91ee-382257a723b3-ovnkube-script-lib\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.467985 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.467740    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6db8c394-338e-4a79-a776-1750a359ffa9-cni-binary-copy\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.467985 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.467774    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2b738474-0882-47d5-91ee-382257a723b3-ovnkube-config\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.467985 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.467793    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-etc-sysconfig\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.468883 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.467824    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5bdbf9db-4a68-4625-adc1-a17b0e04d581-host\") pod \"node-ca-cdmwn\" (UID: \"5bdbf9db-4a68-4625-adc1-a17b0e04d581\") " pod="openshift-image-registry/node-ca-cdmwn"
Jul 30 19:24:33.468883 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.467983    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d7b84b62-b1c3-4ad0-a65b-e8b1238498d1-etc-selinux\") pod \"aws-ebs-csi-driver-node-gvfg8\" (UID: \"d7b84b62-b1c3-4ad0-a65b-e8b1238498d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8"
Jul 30 19:24:33.468883 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468025    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3ad1522f-8aea-4551-af0c-131bf223b514-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5j6j7\" (UID: \"3ad1522f-8aea-4551-af0c-131bf223b514\") " pod="openshift-multus/multus-additional-cni-plugins-5j6j7"
Jul 30 19:24:33.468883 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468076    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-host-var-lib-cni-multus\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.468883 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468109    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbc5h\" (UniqueName: \"kubernetes.io/projected/aec35b15-c096-4e99-a7dc-77be12a9ceca-kube-api-access-gbc5h\") pod \"network-metrics-daemon-dtpk6\" (UID: \"aec35b15-c096-4e99-a7dc-77be12a9ceca\") " pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:24:33.468883 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468177    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3ad1522f-8aea-4551-af0c-131bf223b514-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5j6j7\" (UID: \"3ad1522f-8aea-4551-af0c-131bf223b514\") " pod="openshift-multus/multus-additional-cni-plugins-5j6j7"
Jul 30 19:24:33.468883 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468194    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-host-var-lib-cni-multus\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.468883 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468244    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6c766173-91ba-4e51-923b-51ec626a0bee-kubelet-config\") pod \"global-pull-secret-syncer-cjhlf\" (UID: \"6c766173-91ba-4e51-923b-51ec626a0bee\") " pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:24:33.468883 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468268    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c766173-91ba-4e51-923b-51ec626a0bee-original-pull-secret\") pod \"global-pull-secret-syncer-cjhlf\" (UID: \"6c766173-91ba-4e51-923b-51ec626a0bee\") " pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:24:33.468883 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468293    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-run\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.468883 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468316    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d7b84b62-b1c3-4ad0-a65b-e8b1238498d1-registration-dir\") pod \"aws-ebs-csi-driver-node-gvfg8\" (UID: \"d7b84b62-b1c3-4ad0-a65b-e8b1238498d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8"
Jul 30 19:24:33.468883 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468337    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3ad1522f-8aea-4551-af0c-131bf223b514-os-release\") pod \"multus-additional-cni-plugins-5j6j7\" (UID: \"3ad1522f-8aea-4551-af0c-131bf223b514\") " pod="openshift-multus/multus-additional-cni-plugins-5j6j7"
Jul 30 19:24:33.468883 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468370    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-multus-cni-dir\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.468883 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468417    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3ad1522f-8aea-4551-af0c-131bf223b514-os-release\") pod \"multus-additional-cni-plugins-5j6j7\" (UID: \"3ad1522f-8aea-4551-af0c-131bf223b514\") " pod="openshift-multus/multus-additional-cni-plugins-5j6j7"
Jul 30 19:24:33.468883 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468419    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6db8c394-338e-4a79-a776-1750a359ffa9-cni-binary-copy\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.468883 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468439    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2b738474-0882-47d5-91ee-382257a723b3-ovnkube-config\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.468883 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468466    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-host-var-lib-cni-bin\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.469697 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468506    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.469697 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468542    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3ad1522f-8aea-4551-af0c-131bf223b514-cnibin\") pod \"multus-additional-cni-plugins-5j6j7\" (UID: \"3ad1522f-8aea-4551-af0c-131bf223b514\") " pod="openshift-multus/multus-additional-cni-plugins-5j6j7"
Jul 30 19:24:33.469697 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468570    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-cnibin\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.469697 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468582    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-multus-cni-dir\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.469697 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468588    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-host-var-lib-cni-bin\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.469697 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468595    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-os-release\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.469697 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468607    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3ad1522f-8aea-4551-af0c-131bf223b514-cnibin\") pod \"multus-additional-cni-plugins-5j6j7\" (UID: \"3ad1522f-8aea-4551-af0c-131bf223b514\") " pod="openshift-multus/multus-additional-cni-plugins-5j6j7"
Jul 30 19:24:33.469697 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468608    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.469697 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468636    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvjgf\" (UniqueName: \"kubernetes.io/projected/6db8c394-338e-4a79-a776-1750a359ffa9-kube-api-access-gvjgf\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.469697 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468657    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-os-release\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.469697 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468672    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aec35b15-c096-4e99-a7dc-77be12a9ceca-metrics-certs\") pod \"network-metrics-daemon-dtpk6\" (UID: \"aec35b15-c096-4e99-a7dc-77be12a9ceca\") " pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:24:33.469697 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468667    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-cnibin\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.469697 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468706    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-host-cni-bin\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.469697 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468743    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6c766173-91ba-4e51-923b-51ec626a0bee-dbus\") pod \"global-pull-secret-syncer-cjhlf\" (UID: \"6c766173-91ba-4e51-923b-51ec626a0bee\") " pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:24:33.469697 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468771    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-etc-kubernetes\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.469697 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468777    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-host-cni-bin\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.469697 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468797    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drdxm\" (UniqueName: \"kubernetes.io/projected/a5396a9a-e676-4ef3-b9a3-48e0236d329a-kube-api-access-drdxm\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.470514 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468824    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57pzf\" (UniqueName: \"kubernetes.io/projected/617b11a2-dd97-4d93-9078-6a1225762223-kube-api-access-57pzf\") pod \"iptables-alerter-crvgj\" (UID: \"617b11a2-dd97-4d93-9078-6a1225762223\") " pod="openshift-network-operator/iptables-alerter-crvgj"
Jul 30 19:24:33.470514 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:33.468843    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 19:24:33.470514 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468873    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6db8c394-338e-4a79-a776-1750a359ffa9-multus-daemon-config\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.470514 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468900    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2b738474-0882-47d5-91ee-382257a723b3-ovn-node-metrics-cert\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.470514 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:33.468957    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aec35b15-c096-4e99-a7dc-77be12a9ceca-metrics-certs podName:aec35b15-c096-4e99-a7dc-77be12a9ceca nodeName:}" failed. No retries permitted until 2026-07-30 19:24:33.968921938 +0000 UTC m=+1.933502999 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aec35b15-c096-4e99-a7dc-77be12a9ceca-metrics-certs") pod "network-metrics-daemon-dtpk6" (UID: "aec35b15-c096-4e99-a7dc-77be12a9ceca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 19:24:33.470514 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.468978    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-var-lib-kubelet\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.470514 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469027    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d7b84b62-b1c3-4ad0-a65b-e8b1238498d1-sys-fs\") pod \"aws-ebs-csi-driver-node-gvfg8\" (UID: \"d7b84b62-b1c3-4ad0-a65b-e8b1238498d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8"
Jul 30 19:24:33.470514 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469055    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-host-var-lib-kubelet\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.470514 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469089    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7pk5\" (UniqueName: \"kubernetes.io/projected/5028ef75-1c24-4d52-8364-8ff8ac66f6cf-kube-api-access-c7pk5\") pod \"node-resolver-lk9m5\" (UID: \"5028ef75-1c24-4d52-8364-8ff8ac66f6cf\") " pod="openshift-dns/node-resolver-lk9m5"
Jul 30 19:24:33.470514 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469118    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-etc-openvswitch\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.470514 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469133    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-host-var-lib-kubelet\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.470514 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469147    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-node-log\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.470514 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469196    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-node-log\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.470514 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469198    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-etc-sysctl-conf\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.470514 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469227    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-etc-openvswitch\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.470514 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469261    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-etc-systemd\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.470514 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469323    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-sys\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.471435 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469349    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ad1522f-8aea-4551-af0c-131bf223b514-system-cni-dir\") pod \"multus-additional-cni-plugins-5j6j7\" (UID: \"3ad1522f-8aea-4551-af0c-131bf223b514\") " pod="openshift-multus/multus-additional-cni-plugins-5j6j7"
Jul 30 19:24:33.471435 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469365    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-host-run-netns\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.471435 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469387    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-run-ovn\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.471435 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469407    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ad1522f-8aea-4551-af0c-131bf223b514-system-cni-dir\") pod \"multus-additional-cni-plugins-5j6j7\" (UID: \"3ad1522f-8aea-4551-af0c-131bf223b514\") " pod="openshift-multus/multus-additional-cni-plugins-5j6j7"
Jul 30 19:24:33.471435 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469412    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6db8c394-338e-4a79-a776-1750a359ffa9-multus-daemon-config\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.471435 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469362    2554 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 30 19:24:33.471435 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469432    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-host-run-netns\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.471435 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469406    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-etc-sysctl-d\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.471435 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469455    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-run-ovn\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.471435 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469472    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/617b11a2-dd97-4d93-9078-6a1225762223-iptables-alerter-script\") pod \"iptables-alerter-crvgj\" (UID: \"617b11a2-dd97-4d93-9078-6a1225762223\") " pod="openshift-network-operator/iptables-alerter-crvgj"
Jul 30 19:24:33.471435 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469492    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c7d624a3-3eef-42fe-b07a-f5518e4b1c32-konnectivity-ca\") pod \"konnectivity-agent-mrw8v\" (UID: \"c7d624a3-3eef-42fe-b07a-f5518e4b1c32\") " pod="kube-system/konnectivity-agent-mrw8v"
Jul 30 19:24:33.471435 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469512    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d7b84b62-b1c3-4ad0-a65b-e8b1238498d1-device-dir\") pod \"aws-ebs-csi-driver-node-gvfg8\" (UID: \"d7b84b62-b1c3-4ad0-a65b-e8b1238498d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8"
Jul 30 19:24:33.471435 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469565    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5028ef75-1c24-4d52-8364-8ff8ac66f6cf-hosts-file\") pod \"node-resolver-lk9m5\" (UID: \"5028ef75-1c24-4d52-8364-8ff8ac66f6cf\") " pod="openshift-dns/node-resolver-lk9m5"
Jul 30 19:24:33.471435 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469592    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3ad1522f-8aea-4551-af0c-131bf223b514-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5j6j7\" (UID: \"3ad1522f-8aea-4551-af0c-131bf223b514\") " pod="openshift-multus/multus-additional-cni-plugins-5j6j7"
Jul 30 19:24:33.471435 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469620    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-var-lib-openvswitch\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.471435 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469639    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5028ef75-1c24-4d52-8364-8ff8ac66f6cf-hosts-file\") pod \"node-resolver-lk9m5\" (UID: \"5028ef75-1c24-4d52-8364-8ff8ac66f6cf\") " pod="openshift-dns/node-resolver-lk9m5"
Jul 30 19:24:33.471435 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469677    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5bdbf9db-4a68-4625-adc1-a17b0e04d581-serviceca\") pod \"node-ca-cdmwn\" (UID: \"5bdbf9db-4a68-4625-adc1-a17b0e04d581\") " pod="openshift-image-registry/node-ca-cdmwn"
Jul 30 19:24:33.472361 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469695    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-var-lib-openvswitch\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.472361 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469698    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l9wb\" (UniqueName: \"kubernetes.io/projected/5bdbf9db-4a68-4625-adc1-a17b0e04d581-kube-api-access-7l9wb\") pod \"node-ca-cdmwn\" (UID: \"5bdbf9db-4a68-4625-adc1-a17b0e04d581\") " pod="openshift-image-registry/node-ca-cdmwn"
Jul 30 19:24:33.472361 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469737    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3ad1522f-8aea-4551-af0c-131bf223b514-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5j6j7\" (UID: \"3ad1522f-8aea-4551-af0c-131bf223b514\") " pod="openshift-multus/multus-additional-cni-plugins-5j6j7"
Jul 30 19:24:33.472361 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469756    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-systemd-units\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.472361 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469801    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-host-run-netns\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.472361 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469835    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-systemd-units\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.472361 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469851    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d7b84b62-b1c3-4ad0-a65b-e8b1238498d1-socket-dir\") pod \"aws-ebs-csi-driver-node-gvfg8\" (UID: \"d7b84b62-b1c3-4ad0-a65b-e8b1238498d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8"
Jul 30 19:24:33.472361 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469887    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-host-run-netns\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.472361 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469879    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv624\" (UniqueName: \"kubernetes.io/projected/2b738474-0882-47d5-91ee-382257a723b3-kube-api-access-jv624\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.472361 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469935    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc4q7\" (UniqueName: \"kubernetes.io/projected/3ad1522f-8aea-4551-af0c-131bf223b514-kube-api-access-jc4q7\") pod \"multus-additional-cni-plugins-5j6j7\" (UID: \"3ad1522f-8aea-4551-af0c-131bf223b514\") " pod="openshift-multus/multus-additional-cni-plugins-5j6j7"
Jul 30 19:24:33.472361 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469958    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3ad1522f-8aea-4551-af0c-131bf223b514-cni-binary-copy\") pod \"multus-additional-cni-plugins-5j6j7\" (UID: \"3ad1522f-8aea-4551-af0c-131bf223b514\") " pod="openshift-multus/multus-additional-cni-plugins-5j6j7"
Jul 30 19:24:33.472361 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.469988    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-system-cni-dir\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.472361 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.470008    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-host-slash\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.472361 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.470032    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-host-cni-netd\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.472361 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.470060    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-etc-modprobe-d\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.472361 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.470083    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-host\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.472361 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.470109    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c7d624a3-3eef-42fe-b07a-f5518e4b1c32-agent-certs\") pod \"konnectivity-agent-mrw8v\" (UID: \"c7d624a3-3eef-42fe-b07a-f5518e4b1c32\") " pod="kube-system/konnectivity-agent-mrw8v"
Jul 30 19:24:33.473203 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.470137    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-etc-kubernetes\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.473203 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.470162    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-system-cni-dir\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.473203 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.470162    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3ad1522f-8aea-4551-af0c-131bf223b514-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5j6j7\" (UID: \"3ad1522f-8aea-4551-af0c-131bf223b514\") " pod="openshift-multus/multus-additional-cni-plugins-5j6j7"
Jul 30 19:24:33.473203 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.470625    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-host-slash\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.473203 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.470661    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6db8c394-338e-4a79-a776-1750a359ffa9-etc-kubernetes\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.473203 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.470666    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3ad1522f-8aea-4551-af0c-131bf223b514-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5j6j7\" (UID: \"3ad1522f-8aea-4551-af0c-131bf223b514\") " pod="openshift-multus/multus-additional-cni-plugins-5j6j7"
Jul 30 19:24:33.473203 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.470705    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2b738474-0882-47d5-91ee-382257a723b3-host-cni-netd\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.473203 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.471032    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3ad1522f-8aea-4551-af0c-131bf223b514-cni-binary-copy\") pod \"multus-additional-cni-plugins-5j6j7\" (UID: \"3ad1522f-8aea-4551-af0c-131bf223b514\") " pod="openshift-multus/multus-additional-cni-plugins-5j6j7"
Jul 30 19:24:33.474152 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.474132    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2b738474-0882-47d5-91ee-382257a723b3-ovn-node-metrics-cert\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.475293 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:33.475274    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 19:24:33.475372 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:33.475297    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 19:24:33.475372 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:33.475308    2554 projected.go:196] Error preparing data for projected volume kube-api-access-mv49g for pod openshift-network-diagnostics/network-check-target-8nv66: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 19:24:33.475449 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:33.475387    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f218f763-1cba-4443-9784-ba8b9452bb6c-kube-api-access-mv49g podName:f218f763-1cba-4443-9784-ba8b9452bb6c nodeName:}" failed. No retries permitted until 2026-07-30 19:24:33.975374146 +0000 UTC m=+1.939955220 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mv49g" (UniqueName: "kubernetes.io/projected/f218f763-1cba-4443-9784-ba8b9452bb6c-kube-api-access-mv49g") pod "network-check-target-8nv66" (UID: "f218f763-1cba-4443-9784-ba8b9452bb6c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 19:24:33.479111 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.479086    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc4q7\" (UniqueName: \"kubernetes.io/projected/3ad1522f-8aea-4551-af0c-131bf223b514-kube-api-access-jc4q7\") pod \"multus-additional-cni-plugins-5j6j7\" (UID: \"3ad1522f-8aea-4551-af0c-131bf223b514\") " pod="openshift-multus/multus-additional-cni-plugins-5j6j7"
Jul 30 19:24:33.479290 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.479117    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7pk5\" (UniqueName: \"kubernetes.io/projected/5028ef75-1c24-4d52-8364-8ff8ac66f6cf-kube-api-access-c7pk5\") pod \"node-resolver-lk9m5\" (UID: \"5028ef75-1c24-4d52-8364-8ff8ac66f6cf\") " pod="openshift-dns/node-resolver-lk9m5"
Jul 30 19:24:33.479290 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.479121    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv624\" (UniqueName: \"kubernetes.io/projected/2b738474-0882-47d5-91ee-382257a723b3-kube-api-access-jv624\") pod \"ovnkube-node-qnhsf\" (UID: \"2b738474-0882-47d5-91ee-382257a723b3\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.479290 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.479162    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvjgf\" (UniqueName: \"kubernetes.io/projected/6db8c394-338e-4a79-a776-1750a359ffa9-kube-api-access-gvjgf\") pod \"multus-pdw24\" (UID: \"6db8c394-338e-4a79-a776-1750a359ffa9\") " pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.479406 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.479328    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbc5h\" (UniqueName: \"kubernetes.io/projected/aec35b15-c096-4e99-a7dc-77be12a9ceca-kube-api-access-gbc5h\") pod \"network-metrics-daemon-dtpk6\" (UID: \"aec35b15-c096-4e99-a7dc-77be12a9ceca\") " pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:24:33.563486 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.563454    2554 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 19:24:33.571162 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571134    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-etc-sysconfig\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.571336 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571168    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5bdbf9db-4a68-4625-adc1-a17b0e04d581-host\") pod \"node-ca-cdmwn\" (UID: \"5bdbf9db-4a68-4625-adc1-a17b0e04d581\") " pod="openshift-image-registry/node-ca-cdmwn"
Jul 30 19:24:33.571336 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571193    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d7b84b62-b1c3-4ad0-a65b-e8b1238498d1-etc-selinux\") pod \"aws-ebs-csi-driver-node-gvfg8\" (UID: \"d7b84b62-b1c3-4ad0-a65b-e8b1238498d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8"
Jul 30 19:24:33.571336 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571219    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6c766173-91ba-4e51-923b-51ec626a0bee-kubelet-config\") pod \"global-pull-secret-syncer-cjhlf\" (UID: \"6c766173-91ba-4e51-923b-51ec626a0bee\") " pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:24:33.571336 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571245    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c766173-91ba-4e51-923b-51ec626a0bee-original-pull-secret\") pod \"global-pull-secret-syncer-cjhlf\" (UID: \"6c766173-91ba-4e51-923b-51ec626a0bee\") " pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:24:33.571336 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571252    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5bdbf9db-4a68-4625-adc1-a17b0e04d581-host\") pod \"node-ca-cdmwn\" (UID: \"5bdbf9db-4a68-4625-adc1-a17b0e04d581\") " pod="openshift-image-registry/node-ca-cdmwn"
Jul 30 19:24:33.571336 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571261    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-etc-sysconfig\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.571336 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571267    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-run\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.571336 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571310    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-run\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.571336 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571321    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d7b84b62-b1c3-4ad0-a65b-e8b1238498d1-registration-dir\") pod \"aws-ebs-csi-driver-node-gvfg8\" (UID: \"d7b84b62-b1c3-4ad0-a65b-e8b1238498d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8"
Jul 30 19:24:33.571336 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571332    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d7b84b62-b1c3-4ad0-a65b-e8b1238498d1-etc-selinux\") pod \"aws-ebs-csi-driver-node-gvfg8\" (UID: \"d7b84b62-b1c3-4ad0-a65b-e8b1238498d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8"
Jul 30 19:24:33.571877 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571371    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6c766173-91ba-4e51-923b-51ec626a0bee-kubelet-config\") pod \"global-pull-secret-syncer-cjhlf\" (UID: \"6c766173-91ba-4e51-923b-51ec626a0bee\") " pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:24:33.571877 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571370    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6c766173-91ba-4e51-923b-51ec626a0bee-dbus\") pod \"global-pull-secret-syncer-cjhlf\" (UID: \"6c766173-91ba-4e51-923b-51ec626a0bee\") " pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:24:33.571877 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:33.571398    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 19:24:33.571877 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571410    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-etc-kubernetes\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.571877 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571427    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drdxm\" (UniqueName: \"kubernetes.io/projected/a5396a9a-e676-4ef3-b9a3-48e0236d329a-kube-api-access-drdxm\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.571877 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571437    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6c766173-91ba-4e51-923b-51ec626a0bee-dbus\") pod \"global-pull-secret-syncer-cjhlf\" (UID: \"6c766173-91ba-4e51-923b-51ec626a0bee\") " pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:24:33.571877 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571425    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d7b84b62-b1c3-4ad0-a65b-e8b1238498d1-registration-dir\") pod \"aws-ebs-csi-driver-node-gvfg8\" (UID: \"d7b84b62-b1c3-4ad0-a65b-e8b1238498d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8"
Jul 30 19:24:33.571877 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:33.571472    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c766173-91ba-4e51-923b-51ec626a0bee-original-pull-secret podName:6c766173-91ba-4e51-923b-51ec626a0bee nodeName:}" failed. No retries permitted until 2026-07-30 19:24:34.071449768 +0000 UTC m=+2.036030829 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c766173-91ba-4e51-923b-51ec626a0bee-original-pull-secret") pod "global-pull-secret-syncer-cjhlf" (UID: "6c766173-91ba-4e51-923b-51ec626a0bee") : object "kube-system"/"original-pull-secret" not registered
Jul 30 19:24:33.571877 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571481    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-etc-kubernetes\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.571877 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571444    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57pzf\" (UniqueName: \"kubernetes.io/projected/617b11a2-dd97-4d93-9078-6a1225762223-kube-api-access-57pzf\") pod \"iptables-alerter-crvgj\" (UID: \"617b11a2-dd97-4d93-9078-6a1225762223\") " pod="openshift-network-operator/iptables-alerter-crvgj"
Jul 30 19:24:33.571877 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571527    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-var-lib-kubelet\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.571877 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571553    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d7b84b62-b1c3-4ad0-a65b-e8b1238498d1-sys-fs\") pod \"aws-ebs-csi-driver-node-gvfg8\" (UID: \"d7b84b62-b1c3-4ad0-a65b-e8b1238498d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8"
Jul 30 19:24:33.571877 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571584    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-etc-sysctl-conf\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.571877 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571589    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-var-lib-kubelet\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.571877 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571609    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-etc-systemd\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.571877 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571634    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-sys\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.571877 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571647    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d7b84b62-b1c3-4ad0-a65b-e8b1238498d1-sys-fs\") pod \"aws-ebs-csi-driver-node-gvfg8\" (UID: \"d7b84b62-b1c3-4ad0-a65b-e8b1238498d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8"
Jul 30 19:24:33.572700 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571664    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-etc-sysctl-d\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.572700 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571677    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-etc-systemd\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.572700 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571693    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/617b11a2-dd97-4d93-9078-6a1225762223-iptables-alerter-script\") pod \"iptables-alerter-crvgj\" (UID: \"617b11a2-dd97-4d93-9078-6a1225762223\") " pod="openshift-network-operator/iptables-alerter-crvgj"
Jul 30 19:24:33.572700 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571698    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-sys\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.572700 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571757    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c7d624a3-3eef-42fe-b07a-f5518e4b1c32-konnectivity-ca\") pod \"konnectivity-agent-mrw8v\" (UID: \"c7d624a3-3eef-42fe-b07a-f5518e4b1c32\") " pod="kube-system/konnectivity-agent-mrw8v"
Jul 30 19:24:33.572700 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571763    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-etc-sysctl-d\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.572700 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571774    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-etc-sysctl-conf\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.572700 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571784    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d7b84b62-b1c3-4ad0-a65b-e8b1238498d1-device-dir\") pod \"aws-ebs-csi-driver-node-gvfg8\" (UID: \"d7b84b62-b1c3-4ad0-a65b-e8b1238498d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8"
Jul 30 19:24:33.572700 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571815    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5bdbf9db-4a68-4625-adc1-a17b0e04d581-serviceca\") pod \"node-ca-cdmwn\" (UID: \"5bdbf9db-4a68-4625-adc1-a17b0e04d581\") " pod="openshift-image-registry/node-ca-cdmwn"
Jul 30 19:24:33.572700 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571844    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l9wb\" (UniqueName: \"kubernetes.io/projected/5bdbf9db-4a68-4625-adc1-a17b0e04d581-kube-api-access-7l9wb\") pod \"node-ca-cdmwn\" (UID: \"5bdbf9db-4a68-4625-adc1-a17b0e04d581\") " pod="openshift-image-registry/node-ca-cdmwn"
Jul 30 19:24:33.572700 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571873    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d7b84b62-b1c3-4ad0-a65b-e8b1238498d1-socket-dir\") pod \"aws-ebs-csi-driver-node-gvfg8\" (UID: \"d7b84b62-b1c3-4ad0-a65b-e8b1238498d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8"
Jul 30 19:24:33.572700 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571899    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d7b84b62-b1c3-4ad0-a65b-e8b1238498d1-device-dir\") pod \"aws-ebs-csi-driver-node-gvfg8\" (UID: \"d7b84b62-b1c3-4ad0-a65b-e8b1238498d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8"
Jul 30 19:24:33.572700 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.571907    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-etc-modprobe-d\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.572700 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.572008    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-etc-modprobe-d\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.572700 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.572018    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-host\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.572700 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.572059    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c7d624a3-3eef-42fe-b07a-f5518e4b1c32-agent-certs\") pod \"konnectivity-agent-mrw8v\" (UID: \"c7d624a3-3eef-42fe-b07a-f5518e4b1c32\") " pod="kube-system/konnectivity-agent-mrw8v"
Jul 30 19:24:33.572700 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.572071    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d7b84b62-b1c3-4ad0-a65b-e8b1238498d1-socket-dir\") pod \"aws-ebs-csi-driver-node-gvfg8\" (UID: \"d7b84b62-b1c3-4ad0-a65b-e8b1238498d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8"
Jul 30 19:24:33.573269 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.572136    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-lib-modules\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.573269 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.572141    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-host\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.573269 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.572170    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a5396a9a-e676-4ef3-b9a3-48e0236d329a-etc-tuned\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.573269 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.572201    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/617b11a2-dd97-4d93-9078-6a1225762223-host-slash\") pod \"iptables-alerter-crvgj\" (UID: \"617b11a2-dd97-4d93-9078-6a1225762223\") " pod="openshift-network-operator/iptables-alerter-crvgj"
Jul 30 19:24:33.573269 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.572231    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d7b84b62-b1c3-4ad0-a65b-e8b1238498d1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gvfg8\" (UID: \"d7b84b62-b1c3-4ad0-a65b-e8b1238498d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8"
Jul 30 19:24:33.573269 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.572262    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a5396a9a-e676-4ef3-b9a3-48e0236d329a-tmp\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.573269 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.572292    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52nn5\" (UniqueName: \"kubernetes.io/projected/d7b84b62-b1c3-4ad0-a65b-e8b1238498d1-kube-api-access-52nn5\") pod \"aws-ebs-csi-driver-node-gvfg8\" (UID: \"d7b84b62-b1c3-4ad0-a65b-e8b1238498d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8"
Jul 30 19:24:33.573269 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.572299    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/617b11a2-dd97-4d93-9078-6a1225762223-host-slash\") pod \"iptables-alerter-crvgj\" (UID: \"617b11a2-dd97-4d93-9078-6a1225762223\") " pod="openshift-network-operator/iptables-alerter-crvgj"
Jul 30 19:24:33.573269 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.572260    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a5396a9a-e676-4ef3-b9a3-48e0236d329a-lib-modules\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.573269 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.572367    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5bdbf9db-4a68-4625-adc1-a17b0e04d581-serviceca\") pod \"node-ca-cdmwn\" (UID: \"5bdbf9db-4a68-4625-adc1-a17b0e04d581\") " pod="openshift-image-registry/node-ca-cdmwn"
Jul 30 19:24:33.573269 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.572381    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c7d624a3-3eef-42fe-b07a-f5518e4b1c32-konnectivity-ca\") pod \"konnectivity-agent-mrw8v\" (UID: \"c7d624a3-3eef-42fe-b07a-f5518e4b1c32\") " pod="kube-system/konnectivity-agent-mrw8v"
Jul 30 19:24:33.573269 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.572403    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/617b11a2-dd97-4d93-9078-6a1225762223-iptables-alerter-script\") pod \"iptables-alerter-crvgj\" (UID: \"617b11a2-dd97-4d93-9078-6a1225762223\") " pod="openshift-network-operator/iptables-alerter-crvgj"
Jul 30 19:24:33.573269 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.572435    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d7b84b62-b1c3-4ad0-a65b-e8b1238498d1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gvfg8\" (UID: \"d7b84b62-b1c3-4ad0-a65b-e8b1238498d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8"
Jul 30 19:24:33.574487 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.574461    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a5396a9a-e676-4ef3-b9a3-48e0236d329a-tmp\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.574587 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.574547    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a5396a9a-e676-4ef3-b9a3-48e0236d329a-etc-tuned\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.574691 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.574674    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c7d624a3-3eef-42fe-b07a-f5518e4b1c32-agent-certs\") pod \"konnectivity-agent-mrw8v\" (UID: \"c7d624a3-3eef-42fe-b07a-f5518e4b1c32\") " pod="kube-system/konnectivity-agent-mrw8v"
Jul 30 19:24:33.579952 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.579925    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52nn5\" (UniqueName: \"kubernetes.io/projected/d7b84b62-b1c3-4ad0-a65b-e8b1238498d1-kube-api-access-52nn5\") pod \"aws-ebs-csi-driver-node-gvfg8\" (UID: \"d7b84b62-b1c3-4ad0-a65b-e8b1238498d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8"
Jul 30 19:24:33.580188 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.580170    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57pzf\" (UniqueName: \"kubernetes.io/projected/617b11a2-dd97-4d93-9078-6a1225762223-kube-api-access-57pzf\") pod \"iptables-alerter-crvgj\" (UID: \"617b11a2-dd97-4d93-9078-6a1225762223\") " pod="openshift-network-operator/iptables-alerter-crvgj"
Jul 30 19:24:33.580409 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.580389    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drdxm\" (UniqueName: \"kubernetes.io/projected/a5396a9a-e676-4ef3-b9a3-48e0236d329a-kube-api-access-drdxm\") pod \"tuned-rzvqk\" (UID: \"a5396a9a-e676-4ef3-b9a3-48e0236d329a\") " pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.580446 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.580410    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l9wb\" (UniqueName: \"kubernetes.io/projected/5bdbf9db-4a68-4625-adc1-a17b0e04d581-kube-api-access-7l9wb\") pod \"node-ca-cdmwn\" (UID: \"5bdbf9db-4a68-4625-adc1-a17b0e04d581\") " pod="openshift-image-registry/node-ca-cdmwn"
Jul 30 19:24:33.618969 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:33.618914    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15c06b079e0c6b47c947023223cc6cbc.slice/crio-fdfed33ffebcce724d3445ac11c69800262948339131828dff2355c3906e3ebb WatchSource:0}: Error finding container fdfed33ffebcce724d3445ac11c69800262948339131828dff2355c3906e3ebb: Status 404 returned error can't find the container with id fdfed33ffebcce724d3445ac11c69800262948339131828dff2355c3906e3ebb
Jul 30 19:24:33.622204 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:33.622175    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae111a0543c1e08f3cabab9cde34d294.slice/crio-7ef0ddbd72ae93ea33e2072cc21cedcd30deece4b6793d16dfa5c4dd85b8e924 WatchSource:0}: Error finding container 7ef0ddbd72ae93ea33e2072cc21cedcd30deece4b6793d16dfa5c4dd85b8e924: Status 404 returned error can't find the container with id 7ef0ddbd72ae93ea33e2072cc21cedcd30deece4b6793d16dfa5c4dd85b8e924
Jul 30 19:24:33.626603 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.626584    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 19:24:33.687740 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.687648    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lk9m5"
Jul 30 19:24:33.695177 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.695147    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5j6j7"
Jul 30 19:24:33.697881 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:33.697853    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5028ef75_1c24_4d52_8364_8ff8ac66f6cf.slice/crio-83c30f7311feb7abd5866e4127f083b4c76232b6984ed72876cafd1c1a30b542 WatchSource:0}: Error finding container 83c30f7311feb7abd5866e4127f083b4c76232b6984ed72876cafd1c1a30b542: Status 404 returned error can't find the container with id 83c30f7311feb7abd5866e4127f083b4c76232b6984ed72876cafd1c1a30b542
Jul 30 19:24:33.704237 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:33.704215    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ad1522f_8aea_4551_af0c_131bf223b514.slice/crio-9b4b8e9f5ecf4b5463b59b34368c6bf6ddf444ea4e0661a2382a06646d19f85c WatchSource:0}: Error finding container 9b4b8e9f5ecf4b5463b59b34368c6bf6ddf444ea4e0661a2382a06646d19f85c: Status 404 returned error can't find the container with id 9b4b8e9f5ecf4b5463b59b34368c6bf6ddf444ea4e0661a2382a06646d19f85c
Jul 30 19:24:33.720686 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.720657    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pdw24"
Jul 30 19:24:33.730737 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:33.730711    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6db8c394_338e_4a79_a776_1750a359ffa9.slice/crio-ea43552c5766b0d1ff4399dae1e6be10509758df4a3c380b829447b2cf7eab01 WatchSource:0}: Error finding container ea43552c5766b0d1ff4399dae1e6be10509758df4a3c380b829447b2cf7eab01: Status 404 returned error can't find the container with id ea43552c5766b0d1ff4399dae1e6be10509758df4a3c380b829447b2cf7eab01
Jul 30 19:24:33.739050 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.738216    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:24:33.747835 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:33.747808    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b738474_0882_47d5_91ee_382257a723b3.slice/crio-10f78f05e2938eddb4112770c6d194252bf6b93569521f8adf300f9bb6fef358 WatchSource:0}: Error finding container 10f78f05e2938eddb4112770c6d194252bf6b93569521f8adf300f9bb6fef358: Status 404 returned error can't find the container with id 10f78f05e2938eddb4112770c6d194252bf6b93569521f8adf300f9bb6fef358
Jul 30 19:24:33.759628 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.759600    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rzvqk"
Jul 30 19:24:33.768340 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:33.768316    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5396a9a_e676_4ef3_b9a3_48e0236d329a.slice/crio-2148c04896e3a04e6956a2981ba746827a3c046793b6c167183c646947c074b1 WatchSource:0}: Error finding container 2148c04896e3a04e6956a2981ba746827a3c046793b6c167183c646947c074b1: Status 404 returned error can't find the container with id 2148c04896e3a04e6956a2981ba746827a3c046793b6c167183c646947c074b1
Jul 30 19:24:33.788927 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.788900    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cdmwn"
Jul 30 19:24:33.798321 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:33.798289    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bdbf9db_4a68_4625_adc1_a17b0e04d581.slice/crio-3efe7dc68f1c02c61234ec6364e6549e94eb357a21aa577cf32cd84cbbcdb276 WatchSource:0}: Error finding container 3efe7dc68f1c02c61234ec6364e6549e94eb357a21aa577cf32cd84cbbcdb276: Status 404 returned error can't find the container with id 3efe7dc68f1c02c61234ec6364e6549e94eb357a21aa577cf32cd84cbbcdb276
Jul 30 19:24:33.810134 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.810106    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-crvgj"
Jul 30 19:24:33.817380 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.817357    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mrw8v"
Jul 30 19:24:33.819593 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:33.819565    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod617b11a2_dd97_4d93_9078_6a1225762223.slice/crio-6e08974f777bdd5bdb63e34c6a52e2dd1b2971eb16a085315c0fefe41fd2b93a WatchSource:0}: Error finding container 6e08974f777bdd5bdb63e34c6a52e2dd1b2971eb16a085315c0fefe41fd2b93a: Status 404 returned error can't find the container with id 6e08974f777bdd5bdb63e34c6a52e2dd1b2971eb16a085315c0fefe41fd2b93a
Jul 30 19:24:33.820976 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.820953    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8"
Jul 30 19:24:33.829498 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:33.829475    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7d624a3_3eef_42fe_b07a_f5518e4b1c32.slice/crio-2559100af00e66842738c47de87a0409110f4fdbd89f996fc08112bef72802f2 WatchSource:0}: Error finding container 2559100af00e66842738c47de87a0409110f4fdbd89f996fc08112bef72802f2: Status 404 returned error can't find the container with id 2559100af00e66842738c47de87a0409110f4fdbd89f996fc08112bef72802f2
Jul 30 19:24:33.831295 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:24:33.831273    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7b84b62_b1c3_4ad0_a65b_e8b1238498d1.slice/crio-58fd0788c4fae1ab5216a006e355bbce15611e1ed36429c18eb2e1650fdd2aca WatchSource:0}: Error finding container 58fd0788c4fae1ab5216a006e355bbce15611e1ed36429c18eb2e1650fdd2aca: Status 404 returned error can't find the container with id 58fd0788c4fae1ab5216a006e355bbce15611e1ed36429c18eb2e1650fdd2aca
Jul 30 19:24:33.975597 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.975507    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aec35b15-c096-4e99-a7dc-77be12a9ceca-metrics-certs\") pod \"network-metrics-daemon-dtpk6\" (UID: \"aec35b15-c096-4e99-a7dc-77be12a9ceca\") " pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:24:33.975597 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:33.975565    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv49g\" (UniqueName: \"kubernetes.io/projected/f218f763-1cba-4443-9784-ba8b9452bb6c-kube-api-access-mv49g\") pod \"network-check-target-8nv66\" (UID: \"f218f763-1cba-4443-9784-ba8b9452bb6c\") " pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:24:33.975848 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:33.975667    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 19:24:33.975848 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:33.975677    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 19:24:33.975848 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:33.975692    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 19:24:33.975848 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:33.975702    2554 projected.go:196] Error preparing data for projected volume kube-api-access-mv49g for pod openshift-network-diagnostics/network-check-target-8nv66: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 19:24:33.975848 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:33.975740    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aec35b15-c096-4e99-a7dc-77be12a9ceca-metrics-certs podName:aec35b15-c096-4e99-a7dc-77be12a9ceca nodeName:}" failed. No retries permitted until 2026-07-30 19:24:34.975719967 +0000 UTC m=+2.940301042 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aec35b15-c096-4e99-a7dc-77be12a9ceca-metrics-certs") pod "network-metrics-daemon-dtpk6" (UID: "aec35b15-c096-4e99-a7dc-77be12a9ceca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 19:24:33.975848 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:33.975759    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f218f763-1cba-4443-9784-ba8b9452bb6c-kube-api-access-mv49g podName:f218f763-1cba-4443-9784-ba8b9452bb6c nodeName:}" failed. No retries permitted until 2026-07-30 19:24:34.975749765 +0000 UTC m=+2.940330825 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mv49g" (UniqueName: "kubernetes.io/projected/f218f763-1cba-4443-9784-ba8b9452bb6c-kube-api-access-mv49g") pod "network-check-target-8nv66" (UID: "f218f763-1cba-4443-9784-ba8b9452bb6c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 19:24:34.075865 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:34.075832    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c766173-91ba-4e51-923b-51ec626a0bee-original-pull-secret\") pod \"global-pull-secret-syncer-cjhlf\" (UID: \"6c766173-91ba-4e51-923b-51ec626a0bee\") " pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:24:34.076066 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:34.075944    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 19:24:34.076066 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:34.076021    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c766173-91ba-4e51-923b-51ec626a0bee-original-pull-secret podName:6c766173-91ba-4e51-923b-51ec626a0bee nodeName:}" failed. No retries permitted until 2026-07-30 19:24:35.076003508 +0000 UTC m=+3.040584578 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c766173-91ba-4e51-923b-51ec626a0bee-original-pull-secret") pod "global-pull-secret-syncer-cjhlf" (UID: "6c766173-91ba-4e51-923b-51ec626a0bee") : object "kube-system"/"original-pull-secret" not registered
Jul 30 19:24:34.244246 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:34.244152    2554 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 19:24:34.395302 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:34.395147    2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 19:19:33 +0000 UTC" deadline="2028-04-09 05:19:06.962369658 +0000 UTC"
Jul 30 19:24:34.395302 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:34.395217    2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14841h54m32.567158588s"
Jul 30 19:24:34.496813 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:34.496619    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8" event={"ID":"d7b84b62-b1c3-4ad0-a65b-e8b1238498d1","Type":"ContainerStarted","Data":"58fd0788c4fae1ab5216a006e355bbce15611e1ed36429c18eb2e1650fdd2aca"}
Jul 30 19:24:34.518050 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:34.518008    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mrw8v" event={"ID":"c7d624a3-3eef-42fe-b07a-f5518e4b1c32","Type":"ContainerStarted","Data":"2559100af00e66842738c47de87a0409110f4fdbd89f996fc08112bef72802f2"}
Jul 30 19:24:34.526934 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:34.526852    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-crvgj" event={"ID":"617b11a2-dd97-4d93-9078-6a1225762223","Type":"ContainerStarted","Data":"6e08974f777bdd5bdb63e34c6a52e2dd1b2971eb16a085315c0fefe41fd2b93a"}
Jul 30 19:24:34.534067 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:34.533914    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf" event={"ID":"2b738474-0882-47d5-91ee-382257a723b3","Type":"ContainerStarted","Data":"10f78f05e2938eddb4112770c6d194252bf6b93569521f8adf300f9bb6fef358"}
Jul 30 19:24:34.536914 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:34.536877    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-180.ec2.internal" event={"ID":"ae111a0543c1e08f3cabab9cde34d294","Type":"ContainerStarted","Data":"7ef0ddbd72ae93ea33e2072cc21cedcd30deece4b6793d16dfa5c4dd85b8e924"}
Jul 30 19:24:34.540104 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:34.540075    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cdmwn" event={"ID":"5bdbf9db-4a68-4625-adc1-a17b0e04d581","Type":"ContainerStarted","Data":"3efe7dc68f1c02c61234ec6364e6549e94eb357a21aa577cf32cd84cbbcdb276"}
Jul 30 19:24:34.543268 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:34.543185    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rzvqk" event={"ID":"a5396a9a-e676-4ef3-b9a3-48e0236d329a","Type":"ContainerStarted","Data":"2148c04896e3a04e6956a2981ba746827a3c046793b6c167183c646947c074b1"}
Jul 30 19:24:34.547015 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:34.546986    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pdw24" event={"ID":"6db8c394-338e-4a79-a776-1750a359ffa9","Type":"ContainerStarted","Data":"ea43552c5766b0d1ff4399dae1e6be10509758df4a3c380b829447b2cf7eab01"}
Jul 30 19:24:34.549714 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:34.549683    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5j6j7" event={"ID":"3ad1522f-8aea-4551-af0c-131bf223b514","Type":"ContainerStarted","Data":"9b4b8e9f5ecf4b5463b59b34368c6bf6ddf444ea4e0661a2382a06646d19f85c"}
Jul 30 19:24:34.556903 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:34.556872    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lk9m5" event={"ID":"5028ef75-1c24-4d52-8364-8ff8ac66f6cf","Type":"ContainerStarted","Data":"83c30f7311feb7abd5866e4127f083b4c76232b6984ed72876cafd1c1a30b542"}
Jul 30 19:24:34.560355 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:34.560326    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-180.ec2.internal" event={"ID":"15c06b079e0c6b47c947023223cc6cbc","Type":"ContainerStarted","Data":"fdfed33ffebcce724d3445ac11c69800262948339131828dff2355c3906e3ebb"}
Jul 30 19:24:34.985772 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:34.985668    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv49g\" (UniqueName: \"kubernetes.io/projected/f218f763-1cba-4443-9784-ba8b9452bb6c-kube-api-access-mv49g\") pod \"network-check-target-8nv66\" (UID: \"f218f763-1cba-4443-9784-ba8b9452bb6c\") " pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:24:34.985772 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:34.985760    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aec35b15-c096-4e99-a7dc-77be12a9ceca-metrics-certs\") pod \"network-metrics-daemon-dtpk6\" (UID: \"aec35b15-c096-4e99-a7dc-77be12a9ceca\") " pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:24:34.986052 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:34.985966    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 19:24:34.986052 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:34.986051    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aec35b15-c096-4e99-a7dc-77be12a9ceca-metrics-certs podName:aec35b15-c096-4e99-a7dc-77be12a9ceca nodeName:}" failed. No retries permitted until 2026-07-30 19:24:36.986031454 +0000 UTC m=+4.950612530 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aec35b15-c096-4e99-a7dc-77be12a9ceca-metrics-certs") pod "network-metrics-daemon-dtpk6" (UID: "aec35b15-c096-4e99-a7dc-77be12a9ceca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 19:24:34.986506 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:34.986485    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 19:24:34.986506 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:34.986510    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 19:24:34.986698 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:34.986523    2554 projected.go:196] Error preparing data for projected volume kube-api-access-mv49g for pod openshift-network-diagnostics/network-check-target-8nv66: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 19:24:34.986698 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:34.986572    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f218f763-1cba-4443-9784-ba8b9452bb6c-kube-api-access-mv49g podName:f218f763-1cba-4443-9784-ba8b9452bb6c nodeName:}" failed. No retries permitted until 2026-07-30 19:24:36.986554422 +0000 UTC m=+4.951135485 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mv49g" (UniqueName: "kubernetes.io/projected/f218f763-1cba-4443-9784-ba8b9452bb6c-kube-api-access-mv49g") pod "network-check-target-8nv66" (UID: "f218f763-1cba-4443-9784-ba8b9452bb6c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 19:24:35.086662 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:35.086608    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c766173-91ba-4e51-923b-51ec626a0bee-original-pull-secret\") pod \"global-pull-secret-syncer-cjhlf\" (UID: \"6c766173-91ba-4e51-923b-51ec626a0bee\") " pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:24:35.086869 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:35.086808    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 19:24:35.086942 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:35.086900    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c766173-91ba-4e51-923b-51ec626a0bee-original-pull-secret podName:6c766173-91ba-4e51-923b-51ec626a0bee nodeName:}" failed. No retries permitted until 2026-07-30 19:24:37.086876771 +0000 UTC m=+5.051457836 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c766173-91ba-4e51-923b-51ec626a0bee-original-pull-secret") pod "global-pull-secret-syncer-cjhlf" (UID: "6c766173-91ba-4e51-923b-51ec626a0bee") : object "kube-system"/"original-pull-secret" not registered
Jul 30 19:24:35.396099 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:35.395972    2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 19:19:33 +0000 UTC" deadline="2028-03-02 15:07:08.186587741 +0000 UTC"
Jul 30 19:24:35.396099 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:35.396011    2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13939h42m32.790582136s"
Jul 30 19:24:35.481612 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:35.480665    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:24:35.481612 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:35.480810    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nv66" podUID="f218f763-1cba-4443-9784-ba8b9452bb6c"
Jul 30 19:24:35.481612 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:35.481295    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:24:35.481612 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:35.481409    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtpk6" podUID="aec35b15-c096-4e99-a7dc-77be12a9ceca"
Jul 30 19:24:35.481612 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:35.481483    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:24:35.481612 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:35.481559    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjhlf" podUID="6c766173-91ba-4e51-923b-51ec626a0bee"
Jul 30 19:24:36.343473 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:36.343441    2554 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 19:24:37.005852 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:37.004769    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aec35b15-c096-4e99-a7dc-77be12a9ceca-metrics-certs\") pod \"network-metrics-daemon-dtpk6\" (UID: \"aec35b15-c096-4e99-a7dc-77be12a9ceca\") " pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:24:37.005852 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:37.004863    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv49g\" (UniqueName: \"kubernetes.io/projected/f218f763-1cba-4443-9784-ba8b9452bb6c-kube-api-access-mv49g\") pod \"network-check-target-8nv66\" (UID: \"f218f763-1cba-4443-9784-ba8b9452bb6c\") " pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:24:37.005852 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:37.005098    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 19:24:37.005852 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:37.005120    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 19:24:37.005852 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:37.005153    2554 projected.go:196] Error preparing data for projected volume kube-api-access-mv49g for pod openshift-network-diagnostics/network-check-target-8nv66: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 19:24:37.005852 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:37.005223    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f218f763-1cba-4443-9784-ba8b9452bb6c-kube-api-access-mv49g podName:f218f763-1cba-4443-9784-ba8b9452bb6c nodeName:}" failed. No retries permitted until 2026-07-30 19:24:41.005203702 +0000 UTC m=+8.969784767 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mv49g" (UniqueName: "kubernetes.io/projected/f218f763-1cba-4443-9784-ba8b9452bb6c-kube-api-access-mv49g") pod "network-check-target-8nv66" (UID: "f218f763-1cba-4443-9784-ba8b9452bb6c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 19:24:37.005852 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:37.005698    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 19:24:37.005852 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:37.005755    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aec35b15-c096-4e99-a7dc-77be12a9ceca-metrics-certs podName:aec35b15-c096-4e99-a7dc-77be12a9ceca nodeName:}" failed. No retries permitted until 2026-07-30 19:24:41.005738041 +0000 UTC m=+8.970319120 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aec35b15-c096-4e99-a7dc-77be12a9ceca-metrics-certs") pod "network-metrics-daemon-dtpk6" (UID: "aec35b15-c096-4e99-a7dc-77be12a9ceca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 19:24:37.110206 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:37.109486    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c766173-91ba-4e51-923b-51ec626a0bee-original-pull-secret\") pod \"global-pull-secret-syncer-cjhlf\" (UID: \"6c766173-91ba-4e51-923b-51ec626a0bee\") " pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:24:37.110206 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:37.109697    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 19:24:37.110206 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:37.109765    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c766173-91ba-4e51-923b-51ec626a0bee-original-pull-secret podName:6c766173-91ba-4e51-923b-51ec626a0bee nodeName:}" failed. No retries permitted until 2026-07-30 19:24:41.109745591 +0000 UTC m=+9.074326672 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c766173-91ba-4e51-923b-51ec626a0bee-original-pull-secret") pod "global-pull-secret-syncer-cjhlf" (UID: "6c766173-91ba-4e51-923b-51ec626a0bee") : object "kube-system"/"original-pull-secret" not registered
Jul 30 19:24:37.480484 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:37.480386    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:24:37.480665 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:37.480524    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nv66" podUID="f218f763-1cba-4443-9784-ba8b9452bb6c"
Jul 30 19:24:37.480665 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:37.480615    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:24:37.480783 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:37.480704    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtpk6" podUID="aec35b15-c096-4e99-a7dc-77be12a9ceca"
Jul 30 19:24:37.480783 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:37.480759    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:24:37.480923 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:37.480826    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjhlf" podUID="6c766173-91ba-4e51-923b-51ec626a0bee"
Jul 30 19:24:39.481180 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:39.480735    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:24:39.481180 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:39.480781    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:24:39.481180 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:39.480851    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjhlf" podUID="6c766173-91ba-4e51-923b-51ec626a0bee"
Jul 30 19:24:39.481180 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:39.481045    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nv66" podUID="f218f763-1cba-4443-9784-ba8b9452bb6c"
Jul 30 19:24:39.481180 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:39.480743    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:24:39.481700 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:39.481191    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtpk6" podUID="aec35b15-c096-4e99-a7dc-77be12a9ceca"
Jul 30 19:24:41.044626 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:41.044587    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv49g\" (UniqueName: \"kubernetes.io/projected/f218f763-1cba-4443-9784-ba8b9452bb6c-kube-api-access-mv49g\") pod \"network-check-target-8nv66\" (UID: \"f218f763-1cba-4443-9784-ba8b9452bb6c\") " pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:24:41.045120 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:41.044680    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aec35b15-c096-4e99-a7dc-77be12a9ceca-metrics-certs\") pod \"network-metrics-daemon-dtpk6\" (UID: \"aec35b15-c096-4e99-a7dc-77be12a9ceca\") " pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:24:41.045120 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:41.044802    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 19:24:41.045120 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:41.044813    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 19:24:41.045120 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:41.044824    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 19:24:41.045120 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:41.044836    2554 projected.go:196] Error preparing data for projected volume kube-api-access-mv49g for pod openshift-network-diagnostics/network-check-target-8nv66: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 19:24:41.045120 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:41.044874    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aec35b15-c096-4e99-a7dc-77be12a9ceca-metrics-certs podName:aec35b15-c096-4e99-a7dc-77be12a9ceca nodeName:}" failed. No retries permitted until 2026-07-30 19:24:49.044852282 +0000 UTC m=+17.009433358 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aec35b15-c096-4e99-a7dc-77be12a9ceca-metrics-certs") pod "network-metrics-daemon-dtpk6" (UID: "aec35b15-c096-4e99-a7dc-77be12a9ceca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 19:24:41.045120 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:41.044892    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f218f763-1cba-4443-9784-ba8b9452bb6c-kube-api-access-mv49g podName:f218f763-1cba-4443-9784-ba8b9452bb6c nodeName:}" failed. No retries permitted until 2026-07-30 19:24:49.044883203 +0000 UTC m=+17.009464266 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mv49g" (UniqueName: "kubernetes.io/projected/f218f763-1cba-4443-9784-ba8b9452bb6c-kube-api-access-mv49g") pod "network-check-target-8nv66" (UID: "f218f763-1cba-4443-9784-ba8b9452bb6c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 19:24:41.145604 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:41.145566    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c766173-91ba-4e51-923b-51ec626a0bee-original-pull-secret\") pod \"global-pull-secret-syncer-cjhlf\" (UID: \"6c766173-91ba-4e51-923b-51ec626a0bee\") " pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:24:41.145788 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:41.145740    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 19:24:41.145848 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:41.145821    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c766173-91ba-4e51-923b-51ec626a0bee-original-pull-secret podName:6c766173-91ba-4e51-923b-51ec626a0bee nodeName:}" failed. No retries permitted until 2026-07-30 19:24:49.145799239 +0000 UTC m=+17.110380318 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c766173-91ba-4e51-923b-51ec626a0bee-original-pull-secret") pod "global-pull-secret-syncer-cjhlf" (UID: "6c766173-91ba-4e51-923b-51ec626a0bee") : object "kube-system"/"original-pull-secret" not registered
Jul 30 19:24:41.481373 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:41.480507    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:24:41.481373 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:41.480547    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:24:41.481373 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:41.480666    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nv66" podUID="f218f763-1cba-4443-9784-ba8b9452bb6c"
Jul 30 19:24:41.481373 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:41.481147    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtpk6" podUID="aec35b15-c096-4e99-a7dc-77be12a9ceca"
Jul 30 19:24:41.481373 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:41.481198    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:24:41.481373 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:41.481281    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjhlf" podUID="6c766173-91ba-4e51-923b-51ec626a0bee"
Jul 30 19:24:43.480850 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:43.480810    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:24:43.481273 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:43.480941    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjhlf" podUID="6c766173-91ba-4e51-923b-51ec626a0bee"
Jul 30 19:24:43.481383 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:43.481361    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:24:43.481482 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:43.481455    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nv66" podUID="f218f763-1cba-4443-9784-ba8b9452bb6c"
Jul 30 19:24:43.481558 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:43.481531    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:24:43.481634 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:43.481614    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtpk6" podUID="aec35b15-c096-4e99-a7dc-77be12a9ceca"
Jul 30 19:24:45.481354 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:45.481311    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:24:45.481354 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:45.481358    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:24:45.481943 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:45.481322    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:24:45.481943 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:45.481461    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtpk6" podUID="aec35b15-c096-4e99-a7dc-77be12a9ceca"
Jul 30 19:24:45.481943 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:45.481536    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjhlf" podUID="6c766173-91ba-4e51-923b-51ec626a0bee"
Jul 30 19:24:45.481943 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:45.481605    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nv66" podUID="f218f763-1cba-4443-9784-ba8b9452bb6c"
Jul 30 19:24:47.480551 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:47.480515    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:24:47.481039 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:47.480515    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:24:47.481039 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:47.480519    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:24:47.481039 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:47.480713    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjhlf" podUID="6c766173-91ba-4e51-923b-51ec626a0bee"
Jul 30 19:24:47.481039 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:47.480618    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nv66" podUID="f218f763-1cba-4443-9784-ba8b9452bb6c"
Jul 30 19:24:47.481039 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:47.480809    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtpk6" podUID="aec35b15-c096-4e99-a7dc-77be12a9ceca"
Jul 30 19:24:49.107772 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:49.107515    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv49g\" (UniqueName: \"kubernetes.io/projected/f218f763-1cba-4443-9784-ba8b9452bb6c-kube-api-access-mv49g\") pod \"network-check-target-8nv66\" (UID: \"f218f763-1cba-4443-9784-ba8b9452bb6c\") " pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:24:49.108220 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:49.107817    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aec35b15-c096-4e99-a7dc-77be12a9ceca-metrics-certs\") pod \"network-metrics-daemon-dtpk6\" (UID: \"aec35b15-c096-4e99-a7dc-77be12a9ceca\") " pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:24:49.108220 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:49.107707    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 19:24:49.108220 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:49.107858    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 19:24:49.108220 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:49.107873    2554 projected.go:196] Error preparing data for projected volume kube-api-access-mv49g for pod openshift-network-diagnostics/network-check-target-8nv66: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 19:24:49.108220 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:49.107914    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 19:24:49.108220 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:49.107931    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f218f763-1cba-4443-9784-ba8b9452bb6c-kube-api-access-mv49g podName:f218f763-1cba-4443-9784-ba8b9452bb6c nodeName:}" failed. No retries permitted until 2026-07-30 19:25:05.107912285 +0000 UTC m=+33.072493366 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mv49g" (UniqueName: "kubernetes.io/projected/f218f763-1cba-4443-9784-ba8b9452bb6c-kube-api-access-mv49g") pod "network-check-target-8nv66" (UID: "f218f763-1cba-4443-9784-ba8b9452bb6c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 19:24:49.108220 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:49.107966    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aec35b15-c096-4e99-a7dc-77be12a9ceca-metrics-certs podName:aec35b15-c096-4e99-a7dc-77be12a9ceca nodeName:}" failed. No retries permitted until 2026-07-30 19:25:05.107940739 +0000 UTC m=+33.072521798 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aec35b15-c096-4e99-a7dc-77be12a9ceca-metrics-certs") pod "network-metrics-daemon-dtpk6" (UID: "aec35b15-c096-4e99-a7dc-77be12a9ceca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 19:24:49.208524 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:49.208488    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c766173-91ba-4e51-923b-51ec626a0bee-original-pull-secret\") pod \"global-pull-secret-syncer-cjhlf\" (UID: \"6c766173-91ba-4e51-923b-51ec626a0bee\") " pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:24:49.208733 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:49.208657    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 19:24:49.208733 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:49.208734    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c766173-91ba-4e51-923b-51ec626a0bee-original-pull-secret podName:6c766173-91ba-4e51-923b-51ec626a0bee nodeName:}" failed. No retries permitted until 2026-07-30 19:25:05.208713894 +0000 UTC m=+33.173294956 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c766173-91ba-4e51-923b-51ec626a0bee-original-pull-secret") pod "global-pull-secret-syncer-cjhlf" (UID: "6c766173-91ba-4e51-923b-51ec626a0bee") : object "kube-system"/"original-pull-secret" not registered
Jul 30 19:24:49.480329 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:49.480292    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:24:49.480533 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:49.480292    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:24:49.480533 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:49.480427    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nv66" podUID="f218f763-1cba-4443-9784-ba8b9452bb6c"
Jul 30 19:24:49.480533 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:49.480292    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:24:49.480533 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:49.480516    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjhlf" podUID="6c766173-91ba-4e51-923b-51ec626a0bee"
Jul 30 19:24:49.480713 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:49.480584    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtpk6" podUID="aec35b15-c096-4e99-a7dc-77be12a9ceca"
Jul 30 19:24:51.481045 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:51.481003    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:24:51.481485 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:51.481009    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:24:51.481485 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:51.481123    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nv66" podUID="f218f763-1cba-4443-9784-ba8b9452bb6c"
Jul 30 19:24:51.481485 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:51.481016    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:24:51.481485 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:51.481243    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtpk6" podUID="aec35b15-c096-4e99-a7dc-77be12a9ceca"
Jul 30 19:24:51.481485 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:51.481287    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjhlf" podUID="6c766173-91ba-4e51-923b-51ec626a0bee"
Jul 30 19:24:53.481163 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:53.481129    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:24:53.481591 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:53.481237    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nv66" podUID="f218f763-1cba-4443-9784-ba8b9452bb6c"
Jul 30 19:24:53.481591 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:53.481139    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:24:53.481591 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:53.481312    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjhlf" podUID="6c766173-91ba-4e51-923b-51ec626a0bee"
Jul 30 19:24:53.481591 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:53.481130    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:24:53.481591 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:53.481388    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtpk6" podUID="aec35b15-c096-4e99-a7dc-77be12a9ceca"
Jul 30 19:24:54.616285 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:54.616065    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rzvqk" event={"ID":"a5396a9a-e676-4ef3-b9a3-48e0236d329a","Type":"ContainerStarted","Data":"5984fdcdc45f3054dabd787a90fd8fca50b26e1804a9a4928c319424cb6ab080"}
Jul 30 19:24:54.624690 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:54.624651    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pdw24" event={"ID":"6db8c394-338e-4a79-a776-1750a359ffa9","Type":"ContainerStarted","Data":"0fb91ec76df39f9aaab423011015fb7cb707c77d045abca8dafdc6bfdc0b6c27"}
Jul 30 19:24:54.628526 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:54.628505    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 19:24:54.628876 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:54.628853    2554 generic.go:356] "Generic (PLEG): container finished" podID="2b738474-0882-47d5-91ee-382257a723b3" containerID="b202be019c799d88b172dea0e944bc92407c3cb47022c86f5c4d2a8dae64ea64" exitCode=1
Jul 30 19:24:54.628985 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:54.628911    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf" event={"ID":"2b738474-0882-47d5-91ee-382257a723b3","Type":"ContainerStarted","Data":"e7dc563a0a7bd5b7c592418bdc930b3438f8f6d39fa3bf4e2ca3a5e8bf2bf1c7"}
Jul 30 19:24:54.629041 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:54.629013    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf" event={"ID":"2b738474-0882-47d5-91ee-382257a723b3","Type":"ContainerStarted","Data":"97dc41bbbfa873dc3651049b4c69e28c8ad1aab7c19ef3a63b4c9ad4b97ba865"}
Jul 30 19:24:54.629100 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:54.629065    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf" event={"ID":"2b738474-0882-47d5-91ee-382257a723b3","Type":"ContainerDied","Data":"b202be019c799d88b172dea0e944bc92407c3cb47022c86f5c4d2a8dae64ea64"}
Jul 30 19:24:54.629181 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:54.629121    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf" event={"ID":"2b738474-0882-47d5-91ee-382257a723b3","Type":"ContainerStarted","Data":"9eea244f28fb9a4cd361e5b50c92b6bc3518fabf4e0c4a202a34baa1cb8a3f56"}
Jul 30 19:24:54.630441 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:54.630416    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-180.ec2.internal" event={"ID":"ae111a0543c1e08f3cabab9cde34d294","Type":"ContainerStarted","Data":"0c18576ae010acd3d11ea78ee6a0f09429e6bce01782914bb30d06427b809221"}
Jul 30 19:24:54.632780 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:54.632742    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rzvqk" podStartSLOduration=1.3790833710000001 podStartE2EDuration="21.632730552s" totalImagesPullingTime="20.253647181s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 19:24:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:24:54.632393445 +0000 UTC m=+22.596974527" watchObservedRunningTime="2026-07-30 19:24:54.632730552 +0000 UTC m=+22.597311632"
Jul 30 19:24:54.648203 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:54.648166    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-pdw24" podStartSLOduration=1.175379702 podStartE2EDuration="21.648157217s" totalImagesPullingTime="20.472777515s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 19:24:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:24:54.647968612 +0000 UTC m=+22.612549685" watchObservedRunningTime="2026-07-30 19:24:54.648157217 +0000 UTC m=+22.612738297"
Jul 30 19:24:55.480845 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:55.480805    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:24:55.481027 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:55.480805    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:24:55.481027 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:55.480922    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjhlf" podUID="6c766173-91ba-4e51-923b-51ec626a0bee"
Jul 30 19:24:55.481112 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:55.481021    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nv66" podUID="f218f763-1cba-4443-9784-ba8b9452bb6c"
Jul 30 19:24:55.481112 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:55.480805    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:24:55.481174 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:55.481108    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtpk6" podUID="aec35b15-c096-4e99-a7dc-77be12a9ceca"
Jul 30 19:24:55.633917 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:55.633879    2554 generic.go:356] "Generic (PLEG): container finished" podID="15c06b079e0c6b47c947023223cc6cbc" containerID="5cc24c2add53c372a7604f193dddaf473bbc0766ef633dacd219662057f8f372" exitCode=0
Jul 30 19:24:55.634656 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:55.633976    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-180.ec2.internal" event={"ID":"15c06b079e0c6b47c947023223cc6cbc","Type":"ContainerDied","Data":"5cc24c2add53c372a7604f193dddaf473bbc0766ef633dacd219662057f8f372"}
Jul 30 19:24:55.635496 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:55.635458    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8" event={"ID":"d7b84b62-b1c3-4ad0-a65b-e8b1238498d1","Type":"ContainerStarted","Data":"c61d59b8ba6b987142c7e1f38437e27993e1aef4bcdde40f72d9448cc1cfee46"}
Jul 30 19:24:55.636929 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:55.636890    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mrw8v" event={"ID":"c7d624a3-3eef-42fe-b07a-f5518e4b1c32","Type":"ContainerStarted","Data":"bb2e0f65258db8e7bee7bc116aac99a542a5f3167aefa6ebaf6e8aba3fe11e0a"}
Jul 30 19:24:55.638323 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:55.638301    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-crvgj" event={"ID":"617b11a2-dd97-4d93-9078-6a1225762223","Type":"ContainerStarted","Data":"a494e4ecbe5ab100aa7cec3984b3cffe8dac9e0778fe62be31454867eb83d657"}
Jul 30 19:24:55.640866 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:55.640849    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 19:24:55.641207 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:55.641191    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf" event={"ID":"2b738474-0882-47d5-91ee-382257a723b3","Type":"ContainerStarted","Data":"2931658426a481d86155a68b423adc6942de93e7775e9c8184ed95b21899f43b"}
Jul 30 19:24:55.641278 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:55.641263    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf" event={"ID":"2b738474-0882-47d5-91ee-382257a723b3","Type":"ContainerStarted","Data":"2474a1b7f5899dd5354889c46582b67c1d601f1d019673e237bcf439655fcaea"}
Jul 30 19:24:55.642452 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:55.642428    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cdmwn" event={"ID":"5bdbf9db-4a68-4625-adc1-a17b0e04d581","Type":"ContainerStarted","Data":"82e789bd218184865b5883b77ef9053f9b87d62c90d6ad71ce2c9e84bc5cf5ff"}
Jul 30 19:24:55.643772 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:55.643754    2554 generic.go:356] "Generic (PLEG): container finished" podID="3ad1522f-8aea-4551-af0c-131bf223b514" containerID="a7a0dd4bda8e2fcef91ef1a75053ae43e1972b43522f9d377eae941b0febb40f" exitCode=0
Jul 30 19:24:55.643894 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:55.643815    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5j6j7" event={"ID":"3ad1522f-8aea-4551-af0c-131bf223b514","Type":"ContainerDied","Data":"a7a0dd4bda8e2fcef91ef1a75053ae43e1972b43522f9d377eae941b0febb40f"}
Jul 30 19:24:55.647059 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:55.647034    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lk9m5" event={"ID":"5028ef75-1c24-4d52-8364-8ff8ac66f6cf","Type":"ContainerStarted","Data":"7a9374d8b3b3b4848861d8e728ad1c0fd7837e654cf9e904d99ca4119f0e0ec4"}
Jul 30 19:24:55.648330 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:55.648295    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-180.ec2.internal" podStartSLOduration=22.648283329 podStartE2EDuration="22.648283329s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 19:24:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:24:54.667737842 +0000 UTC m=+22.632318924" watchObservedRunningTime="2026-07-30 19:24:55.648283329 +0000 UTC m=+23.612864410"
Jul 30 19:24:55.682538 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:55.682259    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-crvgj" podStartSLOduration=2.493452846 podStartE2EDuration="22.682243325s" totalImagesPullingTime="20.188790479s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 19:24:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:24:55.661478843 +0000 UTC m=+23.626059936" watchObservedRunningTime="2026-07-30 19:24:55.682243325 +0000 UTC m=+23.646824405"
Jul 30 19:24:55.695834 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:55.695775    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-cdmwn" podStartSLOduration=2.484280374 podStartE2EDuration="22.695749949s" totalImagesPullingTime="20.211469575s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 19:24:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:24:55.695642093 +0000 UTC m=+23.660223166" watchObservedRunningTime="2026-07-30 19:24:55.695749949 +0000 UTC m=+23.660331040"
Jul 30 19:24:55.722984 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:55.722925    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mrw8v" podStartSLOduration=10.325129602 podStartE2EDuration="22.722906805s" totalImagesPullingTime="12.397777203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:24:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:24:55.709046873 +0000 UTC m=+23.673627954" watchObservedRunningTime="2026-07-30 19:24:55.722906805 +0000 UTC m=+23.687487885"
Jul 30 19:24:55.723119 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:55.723096    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lk9m5" podStartSLOduration=2.401728941 podStartE2EDuration="22.723088439s" totalImagesPullingTime="20.321359498s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 19:24:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:24:55.722641109 +0000 UTC m=+23.687222189" watchObservedRunningTime="2026-07-30 19:24:55.723088439 +0000 UTC m=+23.687669520"
Jul 30 19:24:56.089970 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:56.089929    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mrw8v"
Jul 30 19:24:56.090564 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:56.090541    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mrw8v"
Jul 30 19:24:56.652673 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:56.652646    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-180.ec2.internal" event={"ID":"15c06b079e0c6b47c947023223cc6cbc","Type":"ContainerStarted","Data":"048d73619ee81e8a597c12cf85ecd1b1b5af2f43df5dd63b52b4eafa27c536ae"}
Jul 30 19:24:56.669165 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:56.669102    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-180.ec2.internal" podStartSLOduration=23.669079676 podStartE2EDuration="23.669079676s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 19:24:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:24:56.668102781 +0000 UTC m=+24.632683862" watchObservedRunningTime="2026-07-30 19:24:56.669079676 +0000 UTC m=+24.633660758"
Jul 30 19:24:56.739012 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:56.738982    2554 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 30 19:24:57.429135 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:57.429028    2554 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-30T19:24:56.739004064Z","UUID":"f46cf77f-6539-47f8-9bb2-15c74195336b","Handler":null,"Name":"","Endpoint":""}
Jul 30 19:24:57.430891 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:57.430867    2554 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 30 19:24:57.431030 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:57.430913    2554 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 30 19:24:57.480751 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:57.480720    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:24:57.480751 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:57.480751    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:24:57.480988 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:57.480854    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nv66" podUID="f218f763-1cba-4443-9784-ba8b9452bb6c"
Jul 30 19:24:57.480988 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:57.480918    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtpk6" podUID="aec35b15-c096-4e99-a7dc-77be12a9ceca"
Jul 30 19:24:57.480988 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:57.480940    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:24:57.481093 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:57.481030    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjhlf" podUID="6c766173-91ba-4e51-923b-51ec626a0bee"
Jul 30 19:24:57.657471 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:57.657435    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8" event={"ID":"d7b84b62-b1c3-4ad0-a65b-e8b1238498d1","Type":"ContainerStarted","Data":"36d3b80af31b8d94cf16a7a2fb4646f0bc1a168c2aece0c9f8306d0cb188acbd"}
Jul 30 19:24:57.662335 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:57.662272    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 19:24:57.662928 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:57.662882    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf" event={"ID":"2b738474-0882-47d5-91ee-382257a723b3","Type":"ContainerStarted","Data":"51891270ae09a9cff16d10169592d781ad0af38d485fddb0d8f10316e744c133"}
Jul 30 19:24:57.663069 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:57.662946    2554 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 30 19:24:58.667039 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:58.667000    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8" event={"ID":"d7b84b62-b1c3-4ad0-a65b-e8b1238498d1","Type":"ContainerStarted","Data":"d24703d764bc3cfe19f569ac9f2b1dea9de4d7dce98433c7aeeaeb7d73ee612a"}
Jul 30 19:24:58.685487 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:58.685431    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvfg8" podStartSLOduration=3.034857856 podStartE2EDuration="25.685414603s" totalImagesPullingTime="22.650556747s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 19:24:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:24:58.684772107 +0000 UTC m=+26.649353187" watchObservedRunningTime="2026-07-30 19:24:58.685414603 +0000 UTC m=+26.649995684"
Jul 30 19:24:59.013015 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:59.012982    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mrw8v"
Jul 30 19:24:59.013245 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:59.013149    2554 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 30 19:24:59.013874 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:59.013737    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mrw8v"
Jul 30 19:24:59.480926 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:59.480741    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:24:59.481115 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:59.480744    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:24:59.481115 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:59.481023    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nv66" podUID="f218f763-1cba-4443-9784-ba8b9452bb6c"
Jul 30 19:24:59.481115 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:59.480741    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:24:59.481115 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:59.481104    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjhlf" podUID="6c766173-91ba-4e51-923b-51ec626a0bee"
Jul 30 19:24:59.481273 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:24:59.481179    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtpk6" podUID="aec35b15-c096-4e99-a7dc-77be12a9ceca"
Jul 30 19:24:59.673795 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:59.673751    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 19:24:59.674364 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:59.674265    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf" event={"ID":"2b738474-0882-47d5-91ee-382257a723b3","Type":"ContainerStarted","Data":"ae051fe3df0a203120c45b656c7c02cf34f01187a895d5cd7e39fe990c727279"}
Jul 30 19:24:59.674828 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:24:59.674804    2554 scope.go:117] "RemoveContainer" containerID="b202be019c799d88b172dea0e944bc92407c3cb47022c86f5c4d2a8dae64ea64"
Jul 30 19:25:00.681326 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:00.681296    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 19:25:00.682103 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:00.681716    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf" event={"ID":"2b738474-0882-47d5-91ee-382257a723b3","Type":"ContainerStarted","Data":"e3819e3e0b51fba431fef03a6ca7568897f420f20245b4d724c128eedaabab50"}
Jul 30 19:25:00.683010 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:00.682988    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:25:00.683118 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:00.683077    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:25:00.683198 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:00.683185    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:25:00.701633 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:00.701535    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:25:00.701795 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:00.701770    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:25:00.711992 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:00.711932    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf" podStartSLOduration=7.385571665 podStartE2EDuration="27.711916614s" totalImagesPullingTime="20.326344949s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 19:24:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:00.710350724 +0000 UTC m=+28.674931805" watchObservedRunningTime="2026-07-30 19:25:00.711916614 +0000 UTC m=+28.676497695"
Jul 30 19:25:01.170223 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:01.170193    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cjhlf"]
Jul 30 19:25:01.170388 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:01.170332    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:25:01.170444 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:01.170422    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjhlf" podUID="6c766173-91ba-4e51-923b-51ec626a0bee"
Jul 30 19:25:01.173604 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:01.173579    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dtpk6"]
Jul 30 19:25:01.173765 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:01.173713    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:25:01.173847 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:01.173825    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtpk6" podUID="aec35b15-c096-4e99-a7dc-77be12a9ceca"
Jul 30 19:25:01.174234 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:01.174212    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8nv66"]
Jul 30 19:25:01.174348 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:01.174306    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:25:01.174406 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:01.174379    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nv66" podUID="f218f763-1cba-4443-9784-ba8b9452bb6c"
Jul 30 19:25:02.481856 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:02.481826    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:25:02.482468 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:02.481941    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjhlf" podUID="6c766173-91ba-4e51-923b-51ec626a0bee"
Jul 30 19:25:02.482468 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:02.481982    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:25:02.482468 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:02.482113    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtpk6" podUID="aec35b15-c096-4e99-a7dc-77be12a9ceca"
Jul 30 19:25:03.480782 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:03.480748    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:25:03.480981 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:03.480891    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nv66" podUID="f218f763-1cba-4443-9784-ba8b9452bb6c"
Jul 30 19:25:04.481264 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:04.481069    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:25:04.481671 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:04.481118    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:25:04.481671 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:04.481344    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtpk6" podUID="aec35b15-c096-4e99-a7dc-77be12a9ceca"
Jul 30 19:25:04.481671 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:04.481394    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjhlf" podUID="6c766173-91ba-4e51-923b-51ec626a0bee"
Jul 30 19:25:04.693605 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:04.693569    2554 generic.go:356] "Generic (PLEG): container finished" podID="3ad1522f-8aea-4551-af0c-131bf223b514" containerID="547585be8da9c599988d98487a31c5040c8ad653102640b7b09052ce16273c06" exitCode=0
Jul 30 19:25:04.693771 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:04.693644    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5j6j7" event={"ID":"3ad1522f-8aea-4551-af0c-131bf223b514","Type":"ContainerDied","Data":"547585be8da9c599988d98487a31c5040c8ad653102640b7b09052ce16273c06"}
Jul 30 19:25:05.119880 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:05.119852    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv49g\" (UniqueName: \"kubernetes.io/projected/f218f763-1cba-4443-9784-ba8b9452bb6c-kube-api-access-mv49g\") pod \"network-check-target-8nv66\" (UID: \"f218f763-1cba-4443-9784-ba8b9452bb6c\") " pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:25:05.120038 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:05.119910    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aec35b15-c096-4e99-a7dc-77be12a9ceca-metrics-certs\") pod \"network-metrics-daemon-dtpk6\" (UID: \"aec35b15-c096-4e99-a7dc-77be12a9ceca\") " pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:25:05.120096 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:05.120028    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 19:25:05.120096 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:05.120051    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 19:25:05.120096 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:05.120077    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 19:25:05.120096 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:05.120092    2554 projected.go:196] Error preparing data for projected volume kube-api-access-mv49g for pod openshift-network-diagnostics/network-check-target-8nv66: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 19:25:05.120253 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:05.120093    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aec35b15-c096-4e99-a7dc-77be12a9ceca-metrics-certs podName:aec35b15-c096-4e99-a7dc-77be12a9ceca nodeName:}" failed. No retries permitted until 2026-07-30 19:25:37.120079461 +0000 UTC m=+65.084660520 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aec35b15-c096-4e99-a7dc-77be12a9ceca-metrics-certs") pod "network-metrics-daemon-dtpk6" (UID: "aec35b15-c096-4e99-a7dc-77be12a9ceca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 19:25:05.120253 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:05.120160    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f218f763-1cba-4443-9784-ba8b9452bb6c-kube-api-access-mv49g podName:f218f763-1cba-4443-9784-ba8b9452bb6c nodeName:}" failed. No retries permitted until 2026-07-30 19:25:37.120142923 +0000 UTC m=+65.084723986 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mv49g" (UniqueName: "kubernetes.io/projected/f218f763-1cba-4443-9784-ba8b9452bb6c-kube-api-access-mv49g") pod "network-check-target-8nv66" (UID: "f218f763-1cba-4443-9784-ba8b9452bb6c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 19:25:05.220706 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:05.220657    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c766173-91ba-4e51-923b-51ec626a0bee-original-pull-secret\") pod \"global-pull-secret-syncer-cjhlf\" (UID: \"6c766173-91ba-4e51-923b-51ec626a0bee\") " pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:25:05.220858 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:05.220826    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 19:25:05.220917 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:05.220902    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c766173-91ba-4e51-923b-51ec626a0bee-original-pull-secret podName:6c766173-91ba-4e51-923b-51ec626a0bee nodeName:}" failed. No retries permitted until 2026-07-30 19:25:37.220884038 +0000 UTC m=+65.185465097 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6c766173-91ba-4e51-923b-51ec626a0bee-original-pull-secret") pod "global-pull-secret-syncer-cjhlf" (UID: "6c766173-91ba-4e51-923b-51ec626a0bee") : object "kube-system"/"original-pull-secret" not registered
Jul 30 19:25:05.481026 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:05.480991    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:25:05.481195 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:05.481114    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nv66" podUID="f218f763-1cba-4443-9784-ba8b9452bb6c"
Jul 30 19:25:06.480631 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:06.480598    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:25:06.480631 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:06.480600    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:25:06.481046 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:06.480761    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cjhlf" podUID="6c766173-91ba-4e51-923b-51ec626a0bee"
Jul 30 19:25:06.481046 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:06.480813    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dtpk6" podUID="aec35b15-c096-4e99-a7dc-77be12a9ceca"
Jul 30 19:25:06.700341 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:06.700307    2554 generic.go:356] "Generic (PLEG): container finished" podID="3ad1522f-8aea-4551-af0c-131bf223b514" containerID="c57f569d68725696d1da1e3043c42b8128f8e42cd5acb3d15f726ab9bc6b0392" exitCode=0
Jul 30 19:25:06.700493 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:06.700356    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5j6j7" event={"ID":"3ad1522f-8aea-4551-af0c-131bf223b514","Type":"ContainerDied","Data":"c57f569d68725696d1da1e3043c42b8128f8e42cd5acb3d15f726ab9bc6b0392"}
Jul 30 19:25:06.869487 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:06.869410    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-180.ec2.internal" event="NodeReady"
Jul 30 19:25:06.869646 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:06.869533    2554 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 30 19:25:06.902912 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:06.902867    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5bcbd966f6-xd72m"]
Jul 30 19:25:06.905994 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:06.905975    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:06.908114 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:06.908093    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 30 19:25:06.908463 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:06.908430    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 30 19:25:06.908563 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:06.908502    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 30 19:25:06.908794 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:06.908774    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9rs47\""
Jul 30 19:25:06.914263 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:06.914238    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 30 19:25:06.915220 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:06.915199    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wh4lj"]
Jul 30 19:25:06.918003 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:06.917983    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wh4lj"
Jul 30 19:25:06.918932 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:06.918913    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bcbd966f6-xd72m"]
Jul 30 19:25:06.919748 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:06.919729    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hqnjj"]
Jul 30 19:25:06.920328 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:06.920302    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 30 19:25:06.920405 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:06.920336    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 30 19:25:06.920542 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:06.920524    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9htd7\""
Jul 30 19:25:06.922365 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:06.922346    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hqnjj"
Jul 30 19:25:06.924308 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:06.924292    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 30 19:25:06.924657 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:06.924639    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8dddr\""
Jul 30 19:25:06.924740 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:06.924692    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 30 19:25:06.924740 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:06.924692    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 30 19:25:06.928822 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:06.928802    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wh4lj"]
Jul 30 19:25:06.931875 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:06.931849    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hqnjj"]
Jul 30 19:25:07.033191 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.033154    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-ca-trust-extracted\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:07.033191 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.033193    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc17657c-96a1-4e8f-b659-dd8a92b29ee0-cert\") pod \"ingress-canary-hqnjj\" (UID: \"cc17657c-96a1-4e8f-b659-dd8a92b29ee0\") " pod="openshift-ingress-canary/ingress-canary-hqnjj"
Jul 30 19:25:07.033411 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.033222    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x9jk\" (UniqueName: \"kubernetes.io/projected/cc17657c-96a1-4e8f-b659-dd8a92b29ee0-kube-api-access-9x9jk\") pod \"ingress-canary-hqnjj\" (UID: \"cc17657c-96a1-4e8f-b659-dd8a92b29ee0\") " pod="openshift-ingress-canary/ingress-canary-hqnjj"
Jul 30 19:25:07.033411 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.033246    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6db7b86-8d56-4418-be35-098fab4e24da-metrics-tls\") pod \"dns-default-wh4lj\" (UID: \"c6db7b86-8d56-4418-be35-098fab4e24da\") " pod="openshift-dns/dns-default-wh4lj"
Jul 30 19:25:07.033411 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.033267    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-image-registry-private-configuration\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:07.033411 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.033304    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-certificates\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:07.033411 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.033339    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-trusted-ca\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:07.033411 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.033367    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-bound-sa-token\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:07.033680 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.033499    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c6db7b86-8d56-4418-be35-098fab4e24da-config-volume\") pod \"dns-default-wh4lj\" (UID: \"c6db7b86-8d56-4418-be35-098fab4e24da\") " pod="openshift-dns/dns-default-wh4lj"
Jul 30 19:25:07.033680 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.033558    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6db7b86-8d56-4418-be35-098fab4e24da-tmp-dir\") pod \"dns-default-wh4lj\" (UID: \"c6db7b86-8d56-4418-be35-098fab4e24da\") " pod="openshift-dns/dns-default-wh4lj"
Jul 30 19:25:07.033680 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.033588    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m9kj\" (UniqueName: \"kubernetes.io/projected/c6db7b86-8d56-4418-be35-098fab4e24da-kube-api-access-8m9kj\") pod \"dns-default-wh4lj\" (UID: \"c6db7b86-8d56-4418-be35-098fab4e24da\") " pod="openshift-dns/dns-default-wh4lj"
Jul 30 19:25:07.033680 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.033616    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-tls\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:07.033680 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.033647    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-installation-pull-secrets\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:07.033680 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.033673    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljcfq\" (UniqueName: \"kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-kube-api-access-ljcfq\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:07.068876 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.068842    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:07.134158 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.134064    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-ca-trust-extracted\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:07.134158 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.134103    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc17657c-96a1-4e8f-b659-dd8a92b29ee0-cert\") pod \"ingress-canary-hqnjj\" (UID: \"cc17657c-96a1-4e8f-b659-dd8a92b29ee0\") " pod="openshift-ingress-canary/ingress-canary-hqnjj"
Jul 30 19:25:07.134158 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.134122    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x9jk\" (UniqueName: \"kubernetes.io/projected/cc17657c-96a1-4e8f-b659-dd8a92b29ee0-kube-api-access-9x9jk\") pod \"ingress-canary-hqnjj\" (UID: \"cc17657c-96a1-4e8f-b659-dd8a92b29ee0\") " pod="openshift-ingress-canary/ingress-canary-hqnjj"
Jul 30 19:25:07.134158 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.134137    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6db7b86-8d56-4418-be35-098fab4e24da-metrics-tls\") pod \"dns-default-wh4lj\" (UID: \"c6db7b86-8d56-4418-be35-098fab4e24da\") " pod="openshift-dns/dns-default-wh4lj"
Jul 30 19:25:07.134158 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.134158    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-image-registry-private-configuration\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:07.134500 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.134198    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-certificates\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:07.134500 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.134223    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-trusted-ca\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:07.134500 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:07.134245    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 19:25:07.134500 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:07.134312    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6db7b86-8d56-4418-be35-098fab4e24da-metrics-tls podName:c6db7b86-8d56-4418-be35-098fab4e24da nodeName:}" failed. No retries permitted until 2026-07-30 19:25:07.634290749 +0000 UTC m=+35.598871823 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c6db7b86-8d56-4418-be35-098fab4e24da-metrics-tls") pod "dns-default-wh4lj" (UID: "c6db7b86-8d56-4418-be35-098fab4e24da") : secret "dns-default-metrics-tls" not found
Jul 30 19:25:07.134500 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.134245    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-bound-sa-token\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:07.134682 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.134541    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c6db7b86-8d56-4418-be35-098fab4e24da-config-volume\") pod \"dns-default-wh4lj\" (UID: \"c6db7b86-8d56-4418-be35-098fab4e24da\") " pod="openshift-dns/dns-default-wh4lj"
Jul 30 19:25:07.134682 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.134595    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6db7b86-8d56-4418-be35-098fab4e24da-tmp-dir\") pod \"dns-default-wh4lj\" (UID: \"c6db7b86-8d56-4418-be35-098fab4e24da\") " pod="openshift-dns/dns-default-wh4lj"
Jul 30 19:25:07.134682 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.134623    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m9kj\" (UniqueName: \"kubernetes.io/projected/c6db7b86-8d56-4418-be35-098fab4e24da-kube-api-access-8m9kj\") pod \"dns-default-wh4lj\" (UID: \"c6db7b86-8d56-4418-be35-098fab4e24da\") " pod="openshift-dns/dns-default-wh4lj"
Jul 30 19:25:07.134682 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.134653    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-ca-trust-extracted\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:07.134797 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.134752    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-tls\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:07.134797 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.134773    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-installation-pull-secrets\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:07.134797 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.134792    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljcfq\" (UniqueName: \"kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-kube-api-access-ljcfq\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:07.134887 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:07.134867    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 19:25:07.134887 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:07.134880    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bcbd966f6-xd72m: secret "image-registry-tls" not found
Jul 30 19:25:07.134986 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:07.134923    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-tls podName:bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf nodeName:}" failed. No retries permitted until 2026-07-30 19:25:07.634908033 +0000 UTC m=+35.599489105 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-tls") pod "image-registry-5bcbd966f6-xd72m" (UID: "bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf") : secret "image-registry-tls" not found
Jul 30 19:25:07.134986 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.134938    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6db7b86-8d56-4418-be35-098fab4e24da-tmp-dir\") pod \"dns-default-wh4lj\" (UID: \"c6db7b86-8d56-4418-be35-098fab4e24da\") " pod="openshift-dns/dns-default-wh4lj"
Jul 30 19:25:07.135251 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.135228    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c6db7b86-8d56-4418-be35-098fab4e24da-config-volume\") pod \"dns-default-wh4lj\" (UID: \"c6db7b86-8d56-4418-be35-098fab4e24da\") " pod="openshift-dns/dns-default-wh4lj"
Jul 30 19:25:07.135311 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.135271    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-certificates\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:07.135718 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.135689    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-trusted-ca\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:07.138711 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.138611    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-image-registry-private-configuration\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:07.138798 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.138656    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc17657c-96a1-4e8f-b659-dd8a92b29ee0-cert\") pod \"ingress-canary-hqnjj\" (UID: \"cc17657c-96a1-4e8f-b659-dd8a92b29ee0\") " pod="openshift-ingress-canary/ingress-canary-hqnjj"
Jul 30 19:25:07.138798 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.138664    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-installation-pull-secrets\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:07.142736 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.142715    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x9jk\" (UniqueName: \"kubernetes.io/projected/cc17657c-96a1-4e8f-b659-dd8a92b29ee0-kube-api-access-9x9jk\") pod \"ingress-canary-hqnjj\" (UID: \"cc17657c-96a1-4e8f-b659-dd8a92b29ee0\") " pod="openshift-ingress-canary/ingress-canary-hqnjj"
Jul 30 19:25:07.142885 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.142866    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-bound-sa-token\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:07.143190 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.143173    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljcfq\" (UniqueName: \"kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-kube-api-access-ljcfq\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:07.144065 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.144042    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m9kj\" (UniqueName: \"kubernetes.io/projected/c6db7b86-8d56-4418-be35-098fab4e24da-kube-api-access-8m9kj\") pod \"dns-default-wh4lj\" (UID: \"c6db7b86-8d56-4418-be35-098fab4e24da\") " pod="openshift-dns/dns-default-wh4lj"
Jul 30 19:25:07.152709 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.152682    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:07.234046 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.234009    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hqnjj"
Jul 30 19:25:07.378969 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.378931    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hqnjj"]
Jul 30 19:25:07.383233 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:25:07.383203    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc17657c_96a1_4e8f_b659_dd8a92b29ee0.slice/crio-2483f4047efb6af3a662e40ccb08266e27ba7f035e2df75e1d2af9bc11e6a9df WatchSource:0}: Error finding container 2483f4047efb6af3a662e40ccb08266e27ba7f035e2df75e1d2af9bc11e6a9df: Status 404 returned error can't find the container with id 2483f4047efb6af3a662e40ccb08266e27ba7f035e2df75e1d2af9bc11e6a9df
Jul 30 19:25:07.481182 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.481149    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:25:07.483746 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.483711    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-82x98\""
Jul 30 19:25:07.483863 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.483805    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 30 19:25:07.483863 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.483847    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 30 19:25:07.638160 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.638080    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-tls\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:07.638283 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.638169    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6db7b86-8d56-4418-be35-098fab4e24da-metrics-tls\") pod \"dns-default-wh4lj\" (UID: \"c6db7b86-8d56-4418-be35-098fab4e24da\") " pod="openshift-dns/dns-default-wh4lj"
Jul 30 19:25:07.638283 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:07.638254    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 19:25:07.638283 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:07.638273    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bcbd966f6-xd72m: secret "image-registry-tls" not found
Jul 30 19:25:07.638283 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:07.638276    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 19:25:07.638409 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:07.638329    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6db7b86-8d56-4418-be35-098fab4e24da-metrics-tls podName:c6db7b86-8d56-4418-be35-098fab4e24da nodeName:}" failed. No retries permitted until 2026-07-30 19:25:08.638314338 +0000 UTC m=+36.602895417 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c6db7b86-8d56-4418-be35-098fab4e24da-metrics-tls") pod "dns-default-wh4lj" (UID: "c6db7b86-8d56-4418-be35-098fab4e24da") : secret "dns-default-metrics-tls" not found
Jul 30 19:25:07.638409 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:07.638342    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-tls podName:bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf nodeName:}" failed. No retries permitted until 2026-07-30 19:25:08.638336734 +0000 UTC m=+36.602917793 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-tls") pod "image-registry-5bcbd966f6-xd72m" (UID: "bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf") : secret "image-registry-tls" not found
Jul 30 19:25:07.704069 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.704031    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hqnjj" event={"ID":"cc17657c-96a1-4e8f-b659-dd8a92b29ee0","Type":"ContainerStarted","Data":"2483f4047efb6af3a662e40ccb08266e27ba7f035e2df75e1d2af9bc11e6a9df"}
Jul 30 19:25:07.706625 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:07.706601    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5j6j7" event={"ID":"3ad1522f-8aea-4551-af0c-131bf223b514","Type":"ContainerStarted","Data":"056a636f33ece2e57688402b583cc1b425b6af1e3df23a6fed4a4538b1442ff2"}
Jul 30 19:25:08.480798 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.480761    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:25:08.481099 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.480761    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:25:08.485175 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.485153    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 19:25:08.485740 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.485622    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bgfkv\""
Jul 30 19:25:08.485740 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.485679    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 19:25:08.647059 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.647016    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-tls\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:08.647059 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.647067    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6db7b86-8d56-4418-be35-098fab4e24da-metrics-tls\") pod \"dns-default-wh4lj\" (UID: \"c6db7b86-8d56-4418-be35-098fab4e24da\") " pod="openshift-dns/dns-default-wh4lj"
Jul 30 19:25:08.647273 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:08.647176    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 19:25:08.647273 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:08.647195    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 19:25:08.647273 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:08.647213    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bcbd966f6-xd72m: secret "image-registry-tls" not found
Jul 30 19:25:08.647273 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:08.647232    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6db7b86-8d56-4418-be35-098fab4e24da-metrics-tls podName:c6db7b86-8d56-4418-be35-098fab4e24da nodeName:}" failed. No retries permitted until 2026-07-30 19:25:10.647216257 +0000 UTC m=+38.611797318 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c6db7b86-8d56-4418-be35-098fab4e24da-metrics-tls") pod "dns-default-wh4lj" (UID: "c6db7b86-8d56-4418-be35-098fab4e24da") : secret "dns-default-metrics-tls" not found
Jul 30 19:25:08.647273 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:08.647256    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-tls podName:bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf nodeName:}" failed. No retries permitted until 2026-07-30 19:25:10.647239903 +0000 UTC m=+38.611820961 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-tls") pod "image-registry-5bcbd966f6-xd72m" (UID: "bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf") : secret "image-registry-tls" not found
Jul 30 19:25:08.713272 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.713238    2554 generic.go:356] "Generic (PLEG): container finished" podID="3ad1522f-8aea-4551-af0c-131bf223b514" containerID="056a636f33ece2e57688402b583cc1b425b6af1e3df23a6fed4a4538b1442ff2" exitCode=0
Jul 30 19:25:08.713459 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.713309    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5j6j7" event={"ID":"3ad1522f-8aea-4551-af0c-131bf223b514","Type":"ContainerDied","Data":"056a636f33ece2e57688402b583cc1b425b6af1e3df23a6fed4a4538b1442ff2"}
Jul 30 19:25:08.725837 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.725797    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-hlccf"]
Jul 30 19:25:08.741821 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.741741    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-hlccf"]
Jul 30 19:25:08.741821 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.741792    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-wrp7d"]
Jul 30 19:25:08.742054 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.741864    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-hlccf"
Jul 30 19:25:08.744891 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.744854    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 30 19:25:08.744891 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.744885    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 30 19:25:08.745119 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.744864    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-2j4xp\""
Jul 30 19:25:08.764083 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.764048    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-msg4s"]
Jul 30 19:25:08.767407 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.765433    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d"
Jul 30 19:25:08.772364 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.768712    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-tjwck\""
Jul 30 19:25:08.772364 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.769015    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 30 19:25:08.772364 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.769263    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 30 19:25:08.772364 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.769820    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 30 19:25:08.772364 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.770099    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 30 19:25:08.777095 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.777071    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 30 19:25:08.782189 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.782164    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-msg4s"]
Jul 30 19:25:08.782189 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.782202    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-wrp7d"]
Jul 30 19:25:08.782368 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.782303    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-msg4s"
Jul 30 19:25:08.784753 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.784732    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-f65h4\""
Jul 30 19:25:08.784928 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.784861    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 30 19:25:08.785021 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.784946    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 30 19:25:08.785021 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.784987    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 30 19:25:08.785135 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.785095    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 30 19:25:08.828589 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.828556    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-77696685b-smtsw"]
Jul 30 19:25:08.840290 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.840254    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-54bd4f7fb-wwb4d"]
Jul 30 19:25:08.840460 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.840427    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-smtsw"
Jul 30 19:25:08.843484 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.843279    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 30 19:25:08.843484 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.843357    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 30 19:25:08.843696 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.843554    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-mch2w\""
Jul 30 19:25:08.843826 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.843807    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 30 19:25:08.844069 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.844033    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 30 19:25:08.848013 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.847946    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6963706-5570-4c07-a0da-707e5bc655a0-trusted-ca\") pod \"console-operator-5ccd6b445f-wrp7d\" (UID: \"a6963706-5570-4c07-a0da-707e5bc655a0\") " pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d"
Jul 30 19:25:08.848147 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.848018    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjr7g\" (UniqueName: \"kubernetes.io/projected/a6963706-5570-4c07-a0da-707e5bc655a0-kube-api-access-cjr7g\") pod \"console-operator-5ccd6b445f-wrp7d\" (UID: \"a6963706-5570-4c07-a0da-707e5bc655a0\") " pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d"
Jul 30 19:25:08.848147 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.848117    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a6963706-5570-4c07-a0da-707e5bc655a0-config\") pod \"console-operator-5ccd6b445f-wrp7d\" (UID: \"a6963706-5570-4c07-a0da-707e5bc655a0\") " pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d"
Jul 30 19:25:08.848252 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.848185    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhw7k\" (UniqueName: \"kubernetes.io/projected/82b54e54-02dd-4fb7-81d2-97b76aa991fe-kube-api-access-zhw7k\") pod \"volume-data-source-validator-5b8776669-hlccf\" (UID: \"82b54e54-02dd-4fb7-81d2-97b76aa991fe\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-hlccf"
Jul 30 19:25:08.848404 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.848384    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a6963706-5570-4c07-a0da-707e5bc655a0-serving-cert\") pod \"console-operator-5ccd6b445f-wrp7d\" (UID: \"a6963706-5570-4c07-a0da-707e5bc655a0\") " pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d"
Jul 30 19:25:08.849353 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.849332    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 30 19:25:08.850381 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.850098    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-smtsw"]
Jul 30 19:25:08.850381 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.850130    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-54bd4f7fb-wwb4d"]
Jul 30 19:25:08.850381 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.850254    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:08.852376 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.852357    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 30 19:25:08.852678 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.852656    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 30 19:25:08.852766 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.852714    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 30 19:25:08.853323 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.853160    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 30 19:25:08.853323 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.853217    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-8pxkx\""
Jul 30 19:25:08.853323 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.853160    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 30 19:25:08.853571 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.853473    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 30 19:25:08.928701 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.928663    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-pgm6p"]
Jul 30 19:25:08.949273 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.949224    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjr7g\" (UniqueName: \"kubernetes.io/projected/a6963706-5570-4c07-a0da-707e5bc655a0-kube-api-access-cjr7g\") pod \"console-operator-5ccd6b445f-wrp7d\" (UID: \"a6963706-5570-4c07-a0da-707e5bc655a0\") " pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d"
Jul 30 19:25:08.949468 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.949282    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mnlz\" (UniqueName: \"kubernetes.io/projected/749026e1-d4ac-408c-b989-e79e0e95306a-kube-api-access-8mnlz\") pod \"router-default-54bd4f7fb-wwb4d\" (UID: \"749026e1-d4ac-408c-b989-e79e0e95306a\") " pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:08.949468 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.949312    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6963706-5570-4c07-a0da-707e5bc655a0-trusted-ca\") pod \"console-operator-5ccd6b445f-wrp7d\" (UID: \"a6963706-5570-4c07-a0da-707e5bc655a0\") " pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d"
Jul 30 19:25:08.949468 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.949346    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9c42650-7ee2-42f7-9b2d-dbf601ba95b3-serving-cert\") pod \"service-ca-operator-784f97bcf6-msg4s\" (UID: \"a9c42650-7ee2-42f7-9b2d-dbf601ba95b3\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-msg4s"
Jul 30 19:25:08.949468 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.949384    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhw7k\" (UniqueName: \"kubernetes.io/projected/82b54e54-02dd-4fb7-81d2-97b76aa991fe-kube-api-access-zhw7k\") pod \"volume-data-source-validator-5b8776669-hlccf\" (UID: \"82b54e54-02dd-4fb7-81d2-97b76aa991fe\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-hlccf"
Jul 30 19:25:08.949468 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.949418    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/471ab30c-b470-404b-b10a-d72642865017-trusted-ca-bundle\") pod \"insights-operator-77696685b-smtsw\" (UID: \"471ab30c-b470-404b-b10a-d72642865017\") " pod="openshift-insights/insights-operator-77696685b-smtsw"
Jul 30 19:25:08.949468 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.949459    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/471ab30c-b470-404b-b10a-d72642865017-snapshots\") pod \"insights-operator-77696685b-smtsw\" (UID: \"471ab30c-b470-404b-b10a-d72642865017\") " pod="openshift-insights/insights-operator-77696685b-smtsw"
Jul 30 19:25:08.949684 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.949491    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/749026e1-d4ac-408c-b989-e79e0e95306a-service-ca-bundle\") pod \"router-default-54bd4f7fb-wwb4d\" (UID: \"749026e1-d4ac-408c-b989-e79e0e95306a\") " pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:08.949684 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.949520    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/749026e1-d4ac-408c-b989-e79e0e95306a-stats-auth\") pod \"router-default-54bd4f7fb-wwb4d\" (UID: \"749026e1-d4ac-408c-b989-e79e0e95306a\") " pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:08.949684 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.949561    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/471ab30c-b470-404b-b10a-d72642865017-tmp\") pod \"insights-operator-77696685b-smtsw\" (UID: \"471ab30c-b470-404b-b10a-d72642865017\") " pod="openshift-insights/insights-operator-77696685b-smtsw"
Jul 30 19:25:08.949684 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.949604    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-869zh\" (UniqueName: \"kubernetes.io/projected/471ab30c-b470-404b-b10a-d72642865017-kube-api-access-869zh\") pod \"insights-operator-77696685b-smtsw\" (UID: \"471ab30c-b470-404b-b10a-d72642865017\") " pod="openshift-insights/insights-operator-77696685b-smtsw"
Jul 30 19:25:08.949684 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.949637    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a6963706-5570-4c07-a0da-707e5bc655a0-config\") pod \"console-operator-5ccd6b445f-wrp7d\" (UID: \"a6963706-5570-4c07-a0da-707e5bc655a0\") " pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d"
Jul 30 19:25:08.949684 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.949661    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/749026e1-d4ac-408c-b989-e79e0e95306a-metrics-certs\") pod \"router-default-54bd4f7fb-wwb4d\" (UID: \"749026e1-d4ac-408c-b989-e79e0e95306a\") " pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:08.949903 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.949721    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9c42650-7ee2-42f7-9b2d-dbf601ba95b3-config\") pod \"service-ca-operator-784f97bcf6-msg4s\" (UID: \"a9c42650-7ee2-42f7-9b2d-dbf601ba95b3\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-msg4s"
Jul 30 19:25:08.949903 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.949749    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/471ab30c-b470-404b-b10a-d72642865017-serving-cert\") pod \"insights-operator-77696685b-smtsw\" (UID: \"471ab30c-b470-404b-b10a-d72642865017\") " pod="openshift-insights/insights-operator-77696685b-smtsw"
Jul 30 19:25:08.949903 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.949775    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plv8b\" (UniqueName: \"kubernetes.io/projected/a9c42650-7ee2-42f7-9b2d-dbf601ba95b3-kube-api-access-plv8b\") pod \"service-ca-operator-784f97bcf6-msg4s\" (UID: \"a9c42650-7ee2-42f7-9b2d-dbf601ba95b3\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-msg4s"
Jul 30 19:25:08.949903 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.949802    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/471ab30c-b470-404b-b10a-d72642865017-service-ca-bundle\") pod \"insights-operator-77696685b-smtsw\" (UID: \"471ab30c-b470-404b-b10a-d72642865017\") " pod="openshift-insights/insights-operator-77696685b-smtsw"
Jul 30 19:25:08.949903 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.949835    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/749026e1-d4ac-408c-b989-e79e0e95306a-default-certificate\") pod \"router-default-54bd4f7fb-wwb4d\" (UID: \"749026e1-d4ac-408c-b989-e79e0e95306a\") " pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:08.949903 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.949867    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a6963706-5570-4c07-a0da-707e5bc655a0-serving-cert\") pod \"console-operator-5ccd6b445f-wrp7d\" (UID: \"a6963706-5570-4c07-a0da-707e5bc655a0\") " pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d"
Jul 30 19:25:08.950852 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.950799    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a6963706-5570-4c07-a0da-707e5bc655a0-config\") pod \"console-operator-5ccd6b445f-wrp7d\" (UID: \"a6963706-5570-4c07-a0da-707e5bc655a0\") " pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d"
Jul 30 19:25:08.951447 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.951182    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6963706-5570-4c07-a0da-707e5bc655a0-trusted-ca\") pod \"console-operator-5ccd6b445f-wrp7d\" (UID: \"a6963706-5570-4c07-a0da-707e5bc655a0\") " pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d"
Jul 30 19:25:08.952292 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.952268    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-72ngp"]
Jul 30 19:25:08.952434 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.952410    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-pgm6p"
Jul 30 19:25:08.955803 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.955777    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-95bxt\""
Jul 30 19:25:08.956327 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.956297    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a6963706-5570-4c07-a0da-707e5bc655a0-serving-cert\") pod \"console-operator-5ccd6b445f-wrp7d\" (UID: \"a6963706-5570-4c07-a0da-707e5bc655a0\") " pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d"
Jul 30 19:25:08.966194 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.966143    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhw7k\" (UniqueName: \"kubernetes.io/projected/82b54e54-02dd-4fb7-81d2-97b76aa991fe-kube-api-access-zhw7k\") pod \"volume-data-source-validator-5b8776669-hlccf\" (UID: \"82b54e54-02dd-4fb7-81d2-97b76aa991fe\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-hlccf"
Jul 30 19:25:08.966194 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.966208    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjr7g\" (UniqueName: \"kubernetes.io/projected/a6963706-5570-4c07-a0da-707e5bc655a0-kube-api-access-cjr7g\") pod \"console-operator-5ccd6b445f-wrp7d\" (UID: \"a6963706-5570-4c07-a0da-707e5bc655a0\") " pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d"
Jul 30 19:25:08.970413 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.970388    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9h8w7"]
Jul 30 19:25:08.970546 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.970530    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-72ngp"
Jul 30 19:25:08.972899 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.972869    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 30 19:25:08.973270 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.973250    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 30 19:25:08.973376 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.973284    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-ngdxk\""
Jul 30 19:25:08.973376 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.973284    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 30 19:25:08.973376 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.973357    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 30 19:25:08.986801 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.986771    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-flnc7"]
Jul 30 19:25:08.986972 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.986883    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9h8w7"
Jul 30 19:25:08.989924 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.989547    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 30 19:25:08.989924 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.989697    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-fm2mn\""
Jul 30 19:25:08.989924 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.989752    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 30 19:25:08.989924 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:08.989833    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 30 19:25:09.008833 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.008789    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-ft7tw"]
Jul 30 19:25:09.009010 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.008936    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-flnc7"
Jul 30 19:25:09.011222 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.011196    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 30 19:25:09.013018 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.012995    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 30 19:25:09.013141 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.013004    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 30 19:25:09.013141 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.013026    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-cgq8c\""
Jul 30 19:25:09.013141 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.013122    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 30 19:25:09.033299 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.033120    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-pgm6p"]
Jul 30 19:25:09.033299 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.033188    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9h8w7"]
Jul 30 19:25:09.033299 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.033188    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-ft7tw"
Jul 30 19:25:09.033299 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.033219    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-ft7tw"]
Jul 30 19:25:09.033299 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.033238    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-72ngp"]
Jul 30 19:25:09.033299 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.033259    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-flnc7"]
Jul 30 19:25:09.036212 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.035947    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-b7hvf\""
Jul 30 19:25:09.036212 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.036200    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 30 19:25:09.036393 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.036221    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 30 19:25:09.050945 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.050913    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9c42650-7ee2-42f7-9b2d-dbf601ba95b3-config\") pod \"service-ca-operator-784f97bcf6-msg4s\" (UID: \"a9c42650-7ee2-42f7-9b2d-dbf601ba95b3\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-msg4s"
Jul 30 19:25:09.050945 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.050946    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/471ab30c-b470-404b-b10a-d72642865017-serving-cert\") pod \"insights-operator-77696685b-smtsw\" (UID: \"471ab30c-b470-404b-b10a-d72642865017\") " pod="openshift-insights/insights-operator-77696685b-smtsw"
Jul 30 19:25:09.051202 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.050987    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plv8b\" (UniqueName: \"kubernetes.io/projected/a9c42650-7ee2-42f7-9b2d-dbf601ba95b3-kube-api-access-plv8b\") pod \"service-ca-operator-784f97bcf6-msg4s\" (UID: \"a9c42650-7ee2-42f7-9b2d-dbf601ba95b3\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-msg4s"
Jul 30 19:25:09.051202 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.051010    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/471ab30c-b470-404b-b10a-d72642865017-service-ca-bundle\") pod \"insights-operator-77696685b-smtsw\" (UID: \"471ab30c-b470-404b-b10a-d72642865017\") " pod="openshift-insights/insights-operator-77696685b-smtsw"
Jul 30 19:25:09.051202 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.051032    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/749026e1-d4ac-408c-b989-e79e0e95306a-default-certificate\") pod \"router-default-54bd4f7fb-wwb4d\" (UID: \"749026e1-d4ac-408c-b989-e79e0e95306a\") " pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:09.051202 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.051060    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/083469f7-657e-4dc8-98ac-70c9749d5922-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-72ngp\" (UID: \"083469f7-657e-4dc8-98ac-70c9749d5922\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-72ngp"
Jul 30 19:25:09.051202 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.051100    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mnlz\" (UniqueName: \"kubernetes.io/projected/749026e1-d4ac-408c-b989-e79e0e95306a-kube-api-access-8mnlz\") pod \"router-default-54bd4f7fb-wwb4d\" (UID: \"749026e1-d4ac-408c-b989-e79e0e95306a\") " pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:09.051202 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.051125    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9c42650-7ee2-42f7-9b2d-dbf601ba95b3-serving-cert\") pod \"service-ca-operator-784f97bcf6-msg4s\" (UID: \"a9c42650-7ee2-42f7-9b2d-dbf601ba95b3\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-msg4s"
Jul 30 19:25:09.051202 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.051152    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4vdf\" (UniqueName: \"kubernetes.io/projected/083469f7-657e-4dc8-98ac-70c9749d5922-kube-api-access-w4vdf\") pod \"kube-storage-version-migrator-operator-7878f4b88c-72ngp\" (UID: \"083469f7-657e-4dc8-98ac-70c9749d5922\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-72ngp"
Jul 30 19:25:09.051202 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.051184    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/471ab30c-b470-404b-b10a-d72642865017-trusted-ca-bundle\") pod \"insights-operator-77696685b-smtsw\" (UID: \"471ab30c-b470-404b-b10a-d72642865017\") " pod="openshift-insights/insights-operator-77696685b-smtsw"
Jul 30 19:25:09.051593 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.051216    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/471ab30c-b470-404b-b10a-d72642865017-snapshots\") pod \"insights-operator-77696685b-smtsw\" (UID: \"471ab30c-b470-404b-b10a-d72642865017\") " pod="openshift-insights/insights-operator-77696685b-smtsw"
Jul 30 19:25:09.051593 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.051246    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/749026e1-d4ac-408c-b989-e79e0e95306a-service-ca-bundle\") pod \"router-default-54bd4f7fb-wwb4d\" (UID: \"749026e1-d4ac-408c-b989-e79e0e95306a\") " pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:09.051593 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.051277    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/749026e1-d4ac-408c-b989-e79e0e95306a-stats-auth\") pod \"router-default-54bd4f7fb-wwb4d\" (UID: \"749026e1-d4ac-408c-b989-e79e0e95306a\") " pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:09.051593 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.051323    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxhqt\" (UniqueName: \"kubernetes.io/projected/6b2c7322-e84b-4ce7-a559-91eb3332fe03-kube-api-access-nxhqt\") pod \"network-check-source-65bdbbff45-pgm6p\" (UID: \"6b2c7322-e84b-4ce7-a559-91eb3332fe03\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-pgm6p"
Jul 30 19:25:09.051593 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.051350    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/471ab30c-b470-404b-b10a-d72642865017-tmp\") pod \"insights-operator-77696685b-smtsw\" (UID: \"471ab30c-b470-404b-b10a-d72642865017\") " pod="openshift-insights/insights-operator-77696685b-smtsw"
Jul 30 19:25:09.051593 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.051392    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-869zh\" (UniqueName: \"kubernetes.io/projected/471ab30c-b470-404b-b10a-d72642865017-kube-api-access-869zh\") pod \"insights-operator-77696685b-smtsw\" (UID: \"471ab30c-b470-404b-b10a-d72642865017\") " pod="openshift-insights/insights-operator-77696685b-smtsw"
Jul 30 19:25:09.051593 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.051428    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/749026e1-d4ac-408c-b989-e79e0e95306a-metrics-certs\") pod \"router-default-54bd4f7fb-wwb4d\" (UID: \"749026e1-d4ac-408c-b989-e79e0e95306a\") " pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:09.051974 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.051668    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/471ab30c-b470-404b-b10a-d72642865017-service-ca-bundle\") pod \"insights-operator-77696685b-smtsw\" (UID: \"471ab30c-b470-404b-b10a-d72642865017\") " pod="openshift-insights/insights-operator-77696685b-smtsw"
Jul 30 19:25:09.051974 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:09.051806    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/749026e1-d4ac-408c-b989-e79e0e95306a-service-ca-bundle podName:749026e1-d4ac-408c-b989-e79e0e95306a nodeName:}" failed. No retries permitted until 2026-07-30 19:25:09.551786863 +0000 UTC m=+37.516367939 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/749026e1-d4ac-408c-b989-e79e0e95306a-service-ca-bundle") pod "router-default-54bd4f7fb-wwb4d" (UID: "749026e1-d4ac-408c-b989-e79e0e95306a") : configmap references non-existent config key: service-ca.crt
Jul 30 19:25:09.052091 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.052028    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/471ab30c-b470-404b-b10a-d72642865017-snapshots\") pod \"insights-operator-77696685b-smtsw\" (UID: \"471ab30c-b470-404b-b10a-d72642865017\") " pod="openshift-insights/insights-operator-77696685b-smtsw"
Jul 30 19:25:09.052146 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.052127    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/471ab30c-b470-404b-b10a-d72642865017-tmp\") pod \"insights-operator-77696685b-smtsw\" (UID: \"471ab30c-b470-404b-b10a-d72642865017\") " pod="openshift-insights/insights-operator-77696685b-smtsw"
Jul 30 19:25:09.052330 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.052305    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/083469f7-657e-4dc8-98ac-70c9749d5922-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-72ngp\" (UID: \"083469f7-657e-4dc8-98ac-70c9749d5922\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-72ngp"
Jul 30 19:25:09.052441 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.052390    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/471ab30c-b470-404b-b10a-d72642865017-trusted-ca-bundle\") pod \"insights-operator-77696685b-smtsw\" (UID: \"471ab30c-b470-404b-b10a-d72642865017\") " pod="openshift-insights/insights-operator-77696685b-smtsw"
Jul 30 19:25:09.052511 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:09.052475    2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 19:25:09.052571 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:09.052540    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/749026e1-d4ac-408c-b989-e79e0e95306a-metrics-certs podName:749026e1-d4ac-408c-b989-e79e0e95306a nodeName:}" failed. No retries permitted until 2026-07-30 19:25:09.552522058 +0000 UTC m=+37.517103131 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/749026e1-d4ac-408c-b989-e79e0e95306a-metrics-certs") pod "router-default-54bd4f7fb-wwb4d" (UID: "749026e1-d4ac-408c-b989-e79e0e95306a") : secret "router-metrics-certs-default" not found
Jul 30 19:25:09.053038 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.052794    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9c42650-7ee2-42f7-9b2d-dbf601ba95b3-config\") pod \"service-ca-operator-784f97bcf6-msg4s\" (UID: \"a9c42650-7ee2-42f7-9b2d-dbf601ba95b3\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-msg4s"
Jul 30 19:25:09.053038 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.052872    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-hlccf"
Jul 30 19:25:09.055055 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.055030    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/471ab30c-b470-404b-b10a-d72642865017-serving-cert\") pod \"insights-operator-77696685b-smtsw\" (UID: \"471ab30c-b470-404b-b10a-d72642865017\") " pod="openshift-insights/insights-operator-77696685b-smtsw"
Jul 30 19:25:09.055327 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.055309    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/749026e1-d4ac-408c-b989-e79e0e95306a-default-certificate\") pod \"router-default-54bd4f7fb-wwb4d\" (UID: \"749026e1-d4ac-408c-b989-e79e0e95306a\") " pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:09.055327 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.055317    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/749026e1-d4ac-408c-b989-e79e0e95306a-stats-auth\") pod \"router-default-54bd4f7fb-wwb4d\" (UID: \"749026e1-d4ac-408c-b989-e79e0e95306a\") " pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:09.060464 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.060413    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9c42650-7ee2-42f7-9b2d-dbf601ba95b3-serving-cert\") pod \"service-ca-operator-784f97bcf6-msg4s\" (UID: \"a9c42650-7ee2-42f7-9b2d-dbf601ba95b3\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-msg4s"
Jul 30 19:25:09.060585 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.060485    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mnlz\" (UniqueName: \"kubernetes.io/projected/749026e1-d4ac-408c-b989-e79e0e95306a-kube-api-access-8mnlz\") pod \"router-default-54bd4f7fb-wwb4d\" (UID: \"749026e1-d4ac-408c-b989-e79e0e95306a\") " pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:09.060903 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.060880    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plv8b\" (UniqueName: \"kubernetes.io/projected/a9c42650-7ee2-42f7-9b2d-dbf601ba95b3-kube-api-access-plv8b\") pod \"service-ca-operator-784f97bcf6-msg4s\" (UID: \"a9c42650-7ee2-42f7-9b2d-dbf601ba95b3\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-msg4s"
Jul 30 19:25:09.060985 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.060933    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-869zh\" (UniqueName: \"kubernetes.io/projected/471ab30c-b470-404b-b10a-d72642865017-kube-api-access-869zh\") pod \"insights-operator-77696685b-smtsw\" (UID: \"471ab30c-b470-404b-b10a-d72642865017\") " pod="openshift-insights/insights-operator-77696685b-smtsw"
Jul 30 19:25:09.083626 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.083602    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d"
Jul 30 19:25:09.092845 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.092817    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-msg4s"
Jul 30 19:25:09.155276 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.154334    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-smtsw"
Jul 30 19:25:09.155276 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.154454    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/083469f7-657e-4dc8-98ac-70c9749d5922-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-72ngp\" (UID: \"083469f7-657e-4dc8-98ac-70c9749d5922\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-72ngp"
Jul 30 19:25:09.155276 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.154501    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ea2bd76b-48c7-417f-ab1e-a1f1414c0aab-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-9h8w7\" (UID: \"ea2bd76b-48c7-417f-ab1e-a1f1414c0aab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9h8w7"
Jul 30 19:25:09.155276 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.154528    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/083469f7-657e-4dc8-98ac-70c9749d5922-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-72ngp\" (UID: \"083469f7-657e-4dc8-98ac-70c9749d5922\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-72ngp"
Jul 30 19:25:09.155276 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.154554    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4vdf\" (UniqueName: \"kubernetes.io/projected/083469f7-657e-4dc8-98ac-70c9749d5922-kube-api-access-w4vdf\") pod \"kube-storage-version-migrator-operator-7878f4b88c-72ngp\" (UID: \"083469f7-657e-4dc8-98ac-70c9749d5922\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-72ngp"
Jul 30 19:25:09.155276 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.154586    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efcb8db8-2128-45ea-9098-ddf248716241-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-ft7tw\" (UID: \"efcb8db8-2128-45ea-9098-ddf248716241\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-ft7tw"
Jul 30 19:25:09.155276 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.154609    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/53668666-0a1b-402a-9c46-dd0835d55256-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-flnc7\" (UID: \"53668666-0a1b-402a-9c46-dd0835d55256\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-flnc7"
Jul 30 19:25:09.155276 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.154652    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knn4t\" (UniqueName: \"kubernetes.io/projected/53668666-0a1b-402a-9c46-dd0835d55256-kube-api-access-knn4t\") pod \"cluster-monitoring-operator-6f55c4f58d-flnc7\" (UID: \"53668666-0a1b-402a-9c46-dd0835d55256\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-flnc7"
Jul 30 19:25:09.155276 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.154678    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxhqt\" (UniqueName: \"kubernetes.io/projected/6b2c7322-e84b-4ce7-a559-91eb3332fe03-kube-api-access-nxhqt\") pod \"network-check-source-65bdbbff45-pgm6p\" (UID: \"6b2c7322-e84b-4ce7-a559-91eb3332fe03\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-pgm6p"
Jul 30 19:25:09.155276 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.154697    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6wfj\" (UniqueName: \"kubernetes.io/projected/ea2bd76b-48c7-417f-ab1e-a1f1414c0aab-kube-api-access-z6wfj\") pod \"cluster-samples-operator-559cbf5745-9h8w7\" (UID: \"ea2bd76b-48c7-417f-ab1e-a1f1414c0aab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9h8w7"
Jul 30 19:25:09.155276 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.154718    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/efcb8db8-2128-45ea-9098-ddf248716241-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-ft7tw\" (UID: \"efcb8db8-2128-45ea-9098-ddf248716241\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-ft7tw"
Jul 30 19:25:09.155276 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.154743    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/53668666-0a1b-402a-9c46-dd0835d55256-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-flnc7\" (UID: \"53668666-0a1b-402a-9c46-dd0835d55256\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-flnc7"
Jul 30 19:25:09.155276 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.155119    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/083469f7-657e-4dc8-98ac-70c9749d5922-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-72ngp\" (UID: \"083469f7-657e-4dc8-98ac-70c9749d5922\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-72ngp"
Jul 30 19:25:09.158311 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.158258    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/083469f7-657e-4dc8-98ac-70c9749d5922-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-72ngp\" (UID: \"083469f7-657e-4dc8-98ac-70c9749d5922\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-72ngp"
Jul 30 19:25:09.163592 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.163531    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4vdf\" (UniqueName: \"kubernetes.io/projected/083469f7-657e-4dc8-98ac-70c9749d5922-kube-api-access-w4vdf\") pod \"kube-storage-version-migrator-operator-7878f4b88c-72ngp\" (UID: \"083469f7-657e-4dc8-98ac-70c9749d5922\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-72ngp"
Jul 30 19:25:09.164782 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.164556    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxhqt\" (UniqueName: \"kubernetes.io/projected/6b2c7322-e84b-4ce7-a559-91eb3332fe03-kube-api-access-nxhqt\") pod \"network-check-source-65bdbbff45-pgm6p\" (UID: \"6b2c7322-e84b-4ce7-a559-91eb3332fe03\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-pgm6p"
Jul 30 19:25:09.164782 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.164699    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:09.253105 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.252985    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-hlccf"]
Jul 30 19:25:09.257366 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.255412    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ea2bd76b-48c7-417f-ab1e-a1f1414c0aab-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-9h8w7\" (UID: \"ea2bd76b-48c7-417f-ab1e-a1f1414c0aab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9h8w7"
Jul 30 19:25:09.257366 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.255488    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efcb8db8-2128-45ea-9098-ddf248716241-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-ft7tw\" (UID: \"efcb8db8-2128-45ea-9098-ddf248716241\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-ft7tw"
Jul 30 19:25:09.257366 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.255524    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/53668666-0a1b-402a-9c46-dd0835d55256-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-flnc7\" (UID: \"53668666-0a1b-402a-9c46-dd0835d55256\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-flnc7"
Jul 30 19:25:09.257366 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.255581    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knn4t\" (UniqueName: \"kubernetes.io/projected/53668666-0a1b-402a-9c46-dd0835d55256-kube-api-access-knn4t\") pod \"cluster-monitoring-operator-6f55c4f58d-flnc7\" (UID: \"53668666-0a1b-402a-9c46-dd0835d55256\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-flnc7"
Jul 30 19:25:09.257366 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.255617    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6wfj\" (UniqueName: \"kubernetes.io/projected/ea2bd76b-48c7-417f-ab1e-a1f1414c0aab-kube-api-access-z6wfj\") pod \"cluster-samples-operator-559cbf5745-9h8w7\" (UID: \"ea2bd76b-48c7-417f-ab1e-a1f1414c0aab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9h8w7"
Jul 30 19:25:09.257366 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.255651    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/efcb8db8-2128-45ea-9098-ddf248716241-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-ft7tw\" (UID: \"efcb8db8-2128-45ea-9098-ddf248716241\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-ft7tw"
Jul 30 19:25:09.257366 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.255702    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/53668666-0a1b-402a-9c46-dd0835d55256-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-flnc7\" (UID: \"53668666-0a1b-402a-9c46-dd0835d55256\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-flnc7"
Jul 30 19:25:09.257366 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.256072    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:09.257366 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:09.256242    2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 30 19:25:09.257366 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:09.256307    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53668666-0a1b-402a-9c46-dd0835d55256-cluster-monitoring-operator-tls podName:53668666-0a1b-402a-9c46-dd0835d55256 nodeName:}" failed. No retries permitted until 2026-07-30 19:25:09.756287706 +0000 UTC m=+37.720868781 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/53668666-0a1b-402a-9c46-dd0835d55256-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-flnc7" (UID: "53668666-0a1b-402a-9c46-dd0835d55256") : secret "cluster-monitoring-operator-tls" not found
Jul 30 19:25:09.257366 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:09.256408    2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 19:25:09.257366 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:09.256451    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efcb8db8-2128-45ea-9098-ddf248716241-networking-console-plugin-cert podName:efcb8db8-2128-45ea-9098-ddf248716241 nodeName:}" failed. No retries permitted until 2026-07-30 19:25:09.756438837 +0000 UTC m=+37.721019907 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/efcb8db8-2128-45ea-9098-ddf248716241-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-ft7tw" (UID: "efcb8db8-2128-45ea-9098-ddf248716241") : secret "networking-console-plugin-cert" not found
Jul 30 19:25:09.257366 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:09.256512    2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 30 19:25:09.257366 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:09.256546    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea2bd76b-48c7-417f-ab1e-a1f1414c0aab-samples-operator-tls podName:ea2bd76b-48c7-417f-ab1e-a1f1414c0aab nodeName:}" failed. No retries permitted until 2026-07-30 19:25:09.75653572 +0000 UTC m=+37.721116788 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ea2bd76b-48c7-417f-ab1e-a1f1414c0aab-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-9h8w7" (UID: "ea2bd76b-48c7-417f-ab1e-a1f1414c0aab") : secret "samples-operator-tls" not found
Jul 30 19:25:09.257366 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.257312    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/efcb8db8-2128-45ea-9098-ddf248716241-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-ft7tw\" (UID: \"efcb8db8-2128-45ea-9098-ddf248716241\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-ft7tw"
Jul 30 19:25:09.259091 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.258753    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/53668666-0a1b-402a-9c46-dd0835d55256-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-flnc7\" (UID: \"53668666-0a1b-402a-9c46-dd0835d55256\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-flnc7"
Jul 30 19:25:09.265331 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.265273    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6wfj\" (UniqueName: \"kubernetes.io/projected/ea2bd76b-48c7-417f-ab1e-a1f1414c0aab-kube-api-access-z6wfj\") pod \"cluster-samples-operator-559cbf5745-9h8w7\" (UID: \"ea2bd76b-48c7-417f-ab1e-a1f1414c0aab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9h8w7"
Jul 30 19:25:09.266986 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.266939    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knn4t\" (UniqueName: \"kubernetes.io/projected/53668666-0a1b-402a-9c46-dd0835d55256-kube-api-access-knn4t\") pod \"cluster-monitoring-operator-6f55c4f58d-flnc7\" (UID: \"53668666-0a1b-402a-9c46-dd0835d55256\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-flnc7"
Jul 30 19:25:09.279586 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.279346    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-pgm6p"
Jul 30 19:25:09.280892 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:25:09.280850    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82b54e54_02dd_4fb7_81d2_97b76aa991fe.slice/crio-60eb384a93947dee50462ee497d49743fbdbf7c556231eb27587c8014a1827e0 WatchSource:0}: Error finding container 60eb384a93947dee50462ee497d49743fbdbf7c556231eb27587c8014a1827e0: Status 404 returned error can't find the container with id 60eb384a93947dee50462ee497d49743fbdbf7c556231eb27587c8014a1827e0
Jul 30 19:25:09.284658 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.284458    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-72ngp"
Jul 30 19:25:09.314639 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.314568    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-wrp7d"]
Jul 30 19:25:09.321794 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:25:09.321754    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6963706_5570_4c07_a0da_707e5bc655a0.slice/crio-9e302ea6e7834761873402b2049d50bec13b4b67332f83d02fe6ef4b33b20f16 WatchSource:0}: Error finding container 9e302ea6e7834761873402b2049d50bec13b4b67332f83d02fe6ef4b33b20f16: Status 404 returned error can't find the container with id 9e302ea6e7834761873402b2049d50bec13b4b67332f83d02fe6ef4b33b20f16
Jul 30 19:25:09.335722 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.335523    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-msg4s"]
Jul 30 19:25:09.338899 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:25:09.338868    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9c42650_7ee2_42f7_9b2d_dbf601ba95b3.slice/crio-ebfee35d8811b4ddb2ade7cd8abe9036cec1e7cd7e22015b5e023e17ce961eb4 WatchSource:0}: Error finding container ebfee35d8811b4ddb2ade7cd8abe9036cec1e7cd7e22015b5e023e17ce961eb4: Status 404 returned error can't find the container with id ebfee35d8811b4ddb2ade7cd8abe9036cec1e7cd7e22015b5e023e17ce961eb4
Jul 30 19:25:09.366968 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.366865    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-smtsw"]
Jul 30 19:25:09.482346 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.481874    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-pgm6p"]
Jul 30 19:25:09.487159 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:25:09.487106    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b2c7322_e84b_4ce7_a559_91eb3332fe03.slice/crio-6fd09275a5a3612d7ca6d19bfb3586ca6470939146867d5e09bcc82c7462178a WatchSource:0}: Error finding container 6fd09275a5a3612d7ca6d19bfb3586ca6470939146867d5e09bcc82c7462178a: Status 404 returned error can't find the container with id 6fd09275a5a3612d7ca6d19bfb3586ca6470939146867d5e09bcc82c7462178a
Jul 30 19:25:09.506068 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.505989    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-72ngp"]
Jul 30 19:25:09.510074 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:25:09.510041    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod083469f7_657e_4dc8_98ac_70c9749d5922.slice/crio-98225987c95baf43567050adfe7c881d47c6545f38e6767232520c928513a192 WatchSource:0}: Error finding container 98225987c95baf43567050adfe7c881d47c6545f38e6767232520c928513a192: Status 404 returned error can't find the container with id 98225987c95baf43567050adfe7c881d47c6545f38e6767232520c928513a192
Jul 30 19:25:09.560390 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.560059    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/749026e1-d4ac-408c-b989-e79e0e95306a-service-ca-bundle\") pod \"router-default-54bd4f7fb-wwb4d\" (UID: \"749026e1-d4ac-408c-b989-e79e0e95306a\") " pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:09.560390 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.560162    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/749026e1-d4ac-408c-b989-e79e0e95306a-metrics-certs\") pod \"router-default-54bd4f7fb-wwb4d\" (UID: \"749026e1-d4ac-408c-b989-e79e0e95306a\") " pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:09.560390 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:09.560223    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/749026e1-d4ac-408c-b989-e79e0e95306a-service-ca-bundle podName:749026e1-d4ac-408c-b989-e79e0e95306a nodeName:}" failed. No retries permitted until 2026-07-30 19:25:10.560198942 +0000 UTC m=+38.524780003 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/749026e1-d4ac-408c-b989-e79e0e95306a-service-ca-bundle") pod "router-default-54bd4f7fb-wwb4d" (UID: "749026e1-d4ac-408c-b989-e79e0e95306a") : configmap references non-existent config key: service-ca.crt
Jul 30 19:25:09.560390 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:09.560284    2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 19:25:09.560390 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:09.560364    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/749026e1-d4ac-408c-b989-e79e0e95306a-metrics-certs podName:749026e1-d4ac-408c-b989-e79e0e95306a nodeName:}" failed. No retries permitted until 2026-07-30 19:25:10.560349043 +0000 UTC m=+38.524930116 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/749026e1-d4ac-408c-b989-e79e0e95306a-metrics-certs") pod "router-default-54bd4f7fb-wwb4d" (UID: "749026e1-d4ac-408c-b989-e79e0e95306a") : secret "router-metrics-certs-default" not found
Jul 30 19:25:09.717191 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.717154    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d" event={"ID":"a6963706-5570-4c07-a0da-707e5bc655a0","Type":"ContainerStarted","Data":"9e302ea6e7834761873402b2049d50bec13b4b67332f83d02fe6ef4b33b20f16"}
Jul 30 19:25:09.718835 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.718801    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-hlccf" event={"ID":"82b54e54-02dd-4fb7-81d2-97b76aa991fe","Type":"ContainerStarted","Data":"60eb384a93947dee50462ee497d49743fbdbf7c556231eb27587c8014a1827e0"}
Jul 30 19:25:09.720537 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.720464    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-72ngp" event={"ID":"083469f7-657e-4dc8-98ac-70c9749d5922","Type":"ContainerStarted","Data":"98225987c95baf43567050adfe7c881d47c6545f38e6767232520c928513a192"}
Jul 30 19:25:09.723014 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.722977    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-smtsw" event={"ID":"471ab30c-b470-404b-b10a-d72642865017","Type":"ContainerStarted","Data":"7cd6efe53da7363e5427e648d969ef6ff01b9c2e26f8b0e5d363c4a60051c28b"}
Jul 30 19:25:09.724738 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.724706    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-msg4s" event={"ID":"a9c42650-7ee2-42f7-9b2d-dbf601ba95b3","Type":"ContainerStarted","Data":"ebfee35d8811b4ddb2ade7cd8abe9036cec1e7cd7e22015b5e023e17ce961eb4"}
Jul 30 19:25:09.729446 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.729418    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-pgm6p" event={"ID":"6b2c7322-e84b-4ce7-a559-91eb3332fe03","Type":"ContainerStarted","Data":"6fd09275a5a3612d7ca6d19bfb3586ca6470939146867d5e09bcc82c7462178a"}
Jul 30 19:25:09.762939 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.762847    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ea2bd76b-48c7-417f-ab1e-a1f1414c0aab-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-9h8w7\" (UID: \"ea2bd76b-48c7-417f-ab1e-a1f1414c0aab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9h8w7"
Jul 30 19:25:09.762939 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.762925    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efcb8db8-2128-45ea-9098-ddf248716241-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-ft7tw\" (UID: \"efcb8db8-2128-45ea-9098-ddf248716241\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-ft7tw"
Jul 30 19:25:09.763204 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:09.763038    2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 19:25:09.763204 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:09.763061    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/53668666-0a1b-402a-9c46-dd0835d55256-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-flnc7\" (UID: \"53668666-0a1b-402a-9c46-dd0835d55256\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-flnc7"
Jul 30 19:25:09.763204 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:09.763038    2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 30 19:25:09.763204 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:09.763107    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efcb8db8-2128-45ea-9098-ddf248716241-networking-console-plugin-cert podName:efcb8db8-2128-45ea-9098-ddf248716241 nodeName:}" failed. No retries permitted until 2026-07-30 19:25:10.763087311 +0000 UTC m=+38.727668381 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/efcb8db8-2128-45ea-9098-ddf248716241-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-ft7tw" (UID: "efcb8db8-2128-45ea-9098-ddf248716241") : secret "networking-console-plugin-cert" not found
Jul 30 19:25:09.763204 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:09.763127    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea2bd76b-48c7-417f-ab1e-a1f1414c0aab-samples-operator-tls podName:ea2bd76b-48c7-417f-ab1e-a1f1414c0aab nodeName:}" failed. No retries permitted until 2026-07-30 19:25:10.763117236 +0000 UTC m=+38.727698314 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ea2bd76b-48c7-417f-ab1e-a1f1414c0aab-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-9h8w7" (UID: "ea2bd76b-48c7-417f-ab1e-a1f1414c0aab") : secret "samples-operator-tls" not found
Jul 30 19:25:09.763474 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:09.763260    2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 30 19:25:09.763474 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:09.763295    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53668666-0a1b-402a-9c46-dd0835d55256-cluster-monitoring-operator-tls podName:53668666-0a1b-402a-9c46-dd0835d55256 nodeName:}" failed. No retries permitted until 2026-07-30 19:25:10.763286293 +0000 UTC m=+38.727867352 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/53668666-0a1b-402a-9c46-dd0835d55256-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-flnc7" (UID: "53668666-0a1b-402a-9c46-dd0835d55256") : secret "cluster-monitoring-operator-tls" not found
Jul 30 19:25:10.572847 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:10.571933    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/749026e1-d4ac-408c-b989-e79e0e95306a-metrics-certs\") pod \"router-default-54bd4f7fb-wwb4d\" (UID: \"749026e1-d4ac-408c-b989-e79e0e95306a\") " pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:10.572847 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:10.572143    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/749026e1-d4ac-408c-b989-e79e0e95306a-service-ca-bundle\") pod \"router-default-54bd4f7fb-wwb4d\" (UID: \"749026e1-d4ac-408c-b989-e79e0e95306a\") " pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:10.572847 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:10.572319    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/749026e1-d4ac-408c-b989-e79e0e95306a-service-ca-bundle podName:749026e1-d4ac-408c-b989-e79e0e95306a nodeName:}" failed. No retries permitted until 2026-07-30 19:25:12.572299338 +0000 UTC m=+40.536880422 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/749026e1-d4ac-408c-b989-e79e0e95306a-service-ca-bundle") pod "router-default-54bd4f7fb-wwb4d" (UID: "749026e1-d4ac-408c-b989-e79e0e95306a") : configmap references non-existent config key: service-ca.crt
Jul 30 19:25:10.572847 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:10.572749    2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 19:25:10.572847 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:10.572804    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/749026e1-d4ac-408c-b989-e79e0e95306a-metrics-certs podName:749026e1-d4ac-408c-b989-e79e0e95306a nodeName:}" failed. No retries permitted until 2026-07-30 19:25:12.572788247 +0000 UTC m=+40.537369326 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/749026e1-d4ac-408c-b989-e79e0e95306a-metrics-certs") pod "router-default-54bd4f7fb-wwb4d" (UID: "749026e1-d4ac-408c-b989-e79e0e95306a") : secret "router-metrics-certs-default" not found
Jul 30 19:25:10.674029 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:10.673177    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-tls\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:10.674029 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:10.673237    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6db7b86-8d56-4418-be35-098fab4e24da-metrics-tls\") pod \"dns-default-wh4lj\" (UID: \"c6db7b86-8d56-4418-be35-098fab4e24da\") " pod="openshift-dns/dns-default-wh4lj"
Jul 30 19:25:10.674029 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:10.673393    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 19:25:10.674029 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:10.673457    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6db7b86-8d56-4418-be35-098fab4e24da-metrics-tls podName:c6db7b86-8d56-4418-be35-098fab4e24da nodeName:}" failed. No retries permitted until 2026-07-30 19:25:14.673437904 +0000 UTC m=+42.638018986 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c6db7b86-8d56-4418-be35-098fab4e24da-metrics-tls") pod "dns-default-wh4lj" (UID: "c6db7b86-8d56-4418-be35-098fab4e24da") : secret "dns-default-metrics-tls" not found
Jul 30 19:25:10.674029 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:10.673863    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 19:25:10.674029 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:10.673880    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bcbd966f6-xd72m: secret "image-registry-tls" not found
Jul 30 19:25:10.674029 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:10.673928    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-tls podName:bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf nodeName:}" failed. No retries permitted until 2026-07-30 19:25:14.673912693 +0000 UTC m=+42.638493768 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-tls") pod "image-registry-5bcbd966f6-xd72m" (UID: "bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf") : secret "image-registry-tls" not found
Jul 30 19:25:10.774512 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:10.774472    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ea2bd76b-48c7-417f-ab1e-a1f1414c0aab-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-9h8w7\" (UID: \"ea2bd76b-48c7-417f-ab1e-a1f1414c0aab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9h8w7"
Jul 30 19:25:10.774680 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:10.774563    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efcb8db8-2128-45ea-9098-ddf248716241-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-ft7tw\" (UID: \"efcb8db8-2128-45ea-9098-ddf248716241\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-ft7tw"
Jul 30 19:25:10.774680 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:10.774602    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/53668666-0a1b-402a-9c46-dd0835d55256-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-flnc7\" (UID: \"53668666-0a1b-402a-9c46-dd0835d55256\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-flnc7"
Jul 30 19:25:10.774797 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:10.774775    2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 30 19:25:10.774901 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:10.774842    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53668666-0a1b-402a-9c46-dd0835d55256-cluster-monitoring-operator-tls podName:53668666-0a1b-402a-9c46-dd0835d55256 nodeName:}" failed. No retries permitted until 2026-07-30 19:25:12.774822642 +0000 UTC m=+40.739403714 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/53668666-0a1b-402a-9c46-dd0835d55256-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-flnc7" (UID: "53668666-0a1b-402a-9c46-dd0835d55256") : secret "cluster-monitoring-operator-tls" not found
Jul 30 19:25:10.775306 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:10.775284    2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 30 19:25:10.775367 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:10.775349    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea2bd76b-48c7-417f-ab1e-a1f1414c0aab-samples-operator-tls podName:ea2bd76b-48c7-417f-ab1e-a1f1414c0aab nodeName:}" failed. No retries permitted until 2026-07-30 19:25:12.77533065 +0000 UTC m=+40.739911726 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ea2bd76b-48c7-417f-ab1e-a1f1414c0aab-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-9h8w7" (UID: "ea2bd76b-48c7-417f-ab1e-a1f1414c0aab") : secret "samples-operator-tls" not found
Jul 30 19:25:10.775431 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:10.775412    2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 19:25:10.775481 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:10.775446    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efcb8db8-2128-45ea-9098-ddf248716241-networking-console-plugin-cert podName:efcb8db8-2128-45ea-9098-ddf248716241 nodeName:}" failed. No retries permitted until 2026-07-30 19:25:12.775435805 +0000 UTC m=+40.740016868 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/efcb8db8-2128-45ea-9098-ddf248716241-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-ft7tw" (UID: "efcb8db8-2128-45ea-9098-ddf248716241") : secret "networking-console-plugin-cert" not found
Jul 30 19:25:11.374255 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:11.374162    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:11.432010 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:11.431972    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:12.358269 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:12.358229    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:12.410204 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:12.410144    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:12.592515 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:12.592478    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/749026e1-d4ac-408c-b989-e79e0e95306a-metrics-certs\") pod \"router-default-54bd4f7fb-wwb4d\" (UID: \"749026e1-d4ac-408c-b989-e79e0e95306a\") " pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:12.592735 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:12.592713    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/749026e1-d4ac-408c-b989-e79e0e95306a-service-ca-bundle\") pod \"router-default-54bd4f7fb-wwb4d\" (UID: \"749026e1-d4ac-408c-b989-e79e0e95306a\") " pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:12.592902 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:12.592884    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/749026e1-d4ac-408c-b989-e79e0e95306a-service-ca-bundle podName:749026e1-d4ac-408c-b989-e79e0e95306a nodeName:}" failed. No retries permitted until 2026-07-30 19:25:16.59286372 +0000 UTC m=+44.557444804 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/749026e1-d4ac-408c-b989-e79e0e95306a-service-ca-bundle") pod "router-default-54bd4f7fb-wwb4d" (UID: "749026e1-d4ac-408c-b989-e79e0e95306a") : configmap references non-existent config key: service-ca.crt
Jul 30 19:25:12.593135 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:12.593091    2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 19:25:12.593215 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:12.593166    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/749026e1-d4ac-408c-b989-e79e0e95306a-metrics-certs podName:749026e1-d4ac-408c-b989-e79e0e95306a nodeName:}" failed. No retries permitted until 2026-07-30 19:25:16.593146455 +0000 UTC m=+44.557727530 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/749026e1-d4ac-408c-b989-e79e0e95306a-metrics-certs") pod "router-default-54bd4f7fb-wwb4d" (UID: "749026e1-d4ac-408c-b989-e79e0e95306a") : secret "router-metrics-certs-default" not found
Jul 30 19:25:12.794257 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:12.794215    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ea2bd76b-48c7-417f-ab1e-a1f1414c0aab-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-9h8w7\" (UID: \"ea2bd76b-48c7-417f-ab1e-a1f1414c0aab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9h8w7"
Jul 30 19:25:12.794449 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:12.794306    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efcb8db8-2128-45ea-9098-ddf248716241-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-ft7tw\" (UID: \"efcb8db8-2128-45ea-9098-ddf248716241\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-ft7tw"
Jul 30 19:25:12.794449 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:12.794347    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/53668666-0a1b-402a-9c46-dd0835d55256-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-flnc7\" (UID: \"53668666-0a1b-402a-9c46-dd0835d55256\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-flnc7"
Jul 30 19:25:12.794580 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:12.794485    2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 30 19:25:12.794580 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:12.794553    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53668666-0a1b-402a-9c46-dd0835d55256-cluster-monitoring-operator-tls podName:53668666-0a1b-402a-9c46-dd0835d55256 nodeName:}" failed. No retries permitted until 2026-07-30 19:25:16.794533732 +0000 UTC m=+44.759114815 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/53668666-0a1b-402a-9c46-dd0835d55256-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-flnc7" (UID: "53668666-0a1b-402a-9c46-dd0835d55256") : secret "cluster-monitoring-operator-tls" not found
Jul 30 19:25:12.794682 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:12.794625    2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 30 19:25:12.794682 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:12.794664    2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 19:25:12.794803 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:12.794667    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea2bd76b-48c7-417f-ab1e-a1f1414c0aab-samples-operator-tls podName:ea2bd76b-48c7-417f-ab1e-a1f1414c0aab nodeName:}" failed. No retries permitted until 2026-07-30 19:25:16.794656268 +0000 UTC m=+44.759237335 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ea2bd76b-48c7-417f-ab1e-a1f1414c0aab-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-9h8w7" (UID: "ea2bd76b-48c7-417f-ab1e-a1f1414c0aab") : secret "samples-operator-tls" not found
Jul 30 19:25:12.794803 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:12.794727    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efcb8db8-2128-45ea-9098-ddf248716241-networking-console-plugin-cert podName:efcb8db8-2128-45ea-9098-ddf248716241 nodeName:}" failed. No retries permitted until 2026-07-30 19:25:16.794715289 +0000 UTC m=+44.759296352 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/efcb8db8-2128-45ea-9098-ddf248716241-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-ft7tw" (UID: "efcb8db8-2128-45ea-9098-ddf248716241") : secret "networking-console-plugin-cert" not found
Jul 30 19:25:14.718827 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:14.718788    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-tls\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:14.719274 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:14.718852    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6db7b86-8d56-4418-be35-098fab4e24da-metrics-tls\") pod \"dns-default-wh4lj\" (UID: \"c6db7b86-8d56-4418-be35-098fab4e24da\") " pod="openshift-dns/dns-default-wh4lj"
Jul 30 19:25:14.719274 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:14.719101    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 19:25:14.719274 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:14.719178    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6db7b86-8d56-4418-be35-098fab4e24da-metrics-tls podName:c6db7b86-8d56-4418-be35-098fab4e24da nodeName:}" failed. No retries permitted until 2026-07-30 19:25:22.719151827 +0000 UTC m=+50.683732909 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c6db7b86-8d56-4418-be35-098fab4e24da-metrics-tls") pod "dns-default-wh4lj" (UID: "c6db7b86-8d56-4418-be35-098fab4e24da") : secret "dns-default-metrics-tls" not found
Jul 30 19:25:14.719695 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:14.719674    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 19:25:14.719695 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:14.719694    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bcbd966f6-xd72m: secret "image-registry-tls" not found
Jul 30 19:25:14.719809 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:14.719746    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-tls podName:bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf nodeName:}" failed. No retries permitted until 2026-07-30 19:25:22.719730405 +0000 UTC m=+50.684311481 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-tls") pod "image-registry-5bcbd966f6-xd72m" (UID: "bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf") : secret "image-registry-tls" not found
Jul 30 19:25:16.636887 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:16.636795    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/749026e1-d4ac-408c-b989-e79e0e95306a-service-ca-bundle\") pod \"router-default-54bd4f7fb-wwb4d\" (UID: \"749026e1-d4ac-408c-b989-e79e0e95306a\") " pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:16.636887 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:16.636880    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/749026e1-d4ac-408c-b989-e79e0e95306a-metrics-certs\") pod \"router-default-54bd4f7fb-wwb4d\" (UID: \"749026e1-d4ac-408c-b989-e79e0e95306a\") " pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:16.637389 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:16.637016    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/749026e1-d4ac-408c-b989-e79e0e95306a-service-ca-bundle podName:749026e1-d4ac-408c-b989-e79e0e95306a nodeName:}" failed. No retries permitted until 2026-07-30 19:25:24.636996205 +0000 UTC m=+52.601577268 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/749026e1-d4ac-408c-b989-e79e0e95306a-service-ca-bundle") pod "router-default-54bd4f7fb-wwb4d" (UID: "749026e1-d4ac-408c-b989-e79e0e95306a") : configmap references non-existent config key: service-ca.crt
Jul 30 19:25:16.637389 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:16.637065    2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 19:25:16.637389 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:16.637138    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/749026e1-d4ac-408c-b989-e79e0e95306a-metrics-certs podName:749026e1-d4ac-408c-b989-e79e0e95306a nodeName:}" failed. No retries permitted until 2026-07-30 19:25:24.637120957 +0000 UTC m=+52.601702034 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/749026e1-d4ac-408c-b989-e79e0e95306a-metrics-certs") pod "router-default-54bd4f7fb-wwb4d" (UID: "749026e1-d4ac-408c-b989-e79e0e95306a") : secret "router-metrics-certs-default" not found
Jul 30 19:25:16.838947 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:16.838908    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ea2bd76b-48c7-417f-ab1e-a1f1414c0aab-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-9h8w7\" (UID: \"ea2bd76b-48c7-417f-ab1e-a1f1414c0aab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9h8w7"
Jul 30 19:25:16.839130 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:16.838981    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efcb8db8-2128-45ea-9098-ddf248716241-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-ft7tw\" (UID: \"efcb8db8-2128-45ea-9098-ddf248716241\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-ft7tw"
Jul 30 19:25:16.839130 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:16.839013    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/53668666-0a1b-402a-9c46-dd0835d55256-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-flnc7\" (UID: \"53668666-0a1b-402a-9c46-dd0835d55256\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-flnc7"
Jul 30 19:25:16.839130 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:16.839098    2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 19:25:16.839130 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:16.839120    2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 30 19:25:16.839316 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:16.839120    2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 30 19:25:16.839316 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:16.839177    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53668666-0a1b-402a-9c46-dd0835d55256-cluster-monitoring-operator-tls podName:53668666-0a1b-402a-9c46-dd0835d55256 nodeName:}" failed. No retries permitted until 2026-07-30 19:25:24.839164109 +0000 UTC m=+52.803745168 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/53668666-0a1b-402a-9c46-dd0835d55256-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-flnc7" (UID: "53668666-0a1b-402a-9c46-dd0835d55256") : secret "cluster-monitoring-operator-tls" not found
Jul 30 19:25:16.839316 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:16.839189    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efcb8db8-2128-45ea-9098-ddf248716241-networking-console-plugin-cert podName:efcb8db8-2128-45ea-9098-ddf248716241 nodeName:}" failed. No retries permitted until 2026-07-30 19:25:24.839183645 +0000 UTC m=+52.803764704 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/efcb8db8-2128-45ea-9098-ddf248716241-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-ft7tw" (UID: "efcb8db8-2128-45ea-9098-ddf248716241") : secret "networking-console-plugin-cert" not found
Jul 30 19:25:16.839316 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:16.839213    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea2bd76b-48c7-417f-ab1e-a1f1414c0aab-samples-operator-tls podName:ea2bd76b-48c7-417f-ab1e-a1f1414c0aab nodeName:}" failed. No retries permitted until 2026-07-30 19:25:24.839193676 +0000 UTC m=+52.803774744 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ea2bd76b-48c7-417f-ab1e-a1f1414c0aab-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-9h8w7" (UID: "ea2bd76b-48c7-417f-ab1e-a1f1414c0aab") : secret "samples-operator-tls" not found
Jul 30 19:25:16.853760 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:16.853729    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:17.027865 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:17.027835    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:17.984630 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:17.984589    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:18.084868 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:18.084826    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:18.138608 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:18.138579    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:18.759808 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:18.759764    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-72ngp" event={"ID":"083469f7-657e-4dc8-98ac-70c9749d5922","Type":"ContainerStarted","Data":"86665d66c58f0994ad816babdae3a412b80241c78b2e6bc74b3802e6028f28c5"}
Jul 30 19:25:18.773980 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:18.773269    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-smtsw" event={"ID":"471ab30c-b470-404b-b10a-d72642865017","Type":"ContainerStarted","Data":"4d68c018c3a86f3cbbf710acfd42d06d52bf11d0888159952f2f93d805ed5612"}
Jul 30 19:25:18.778982 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:18.778746    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5j6j7" event={"ID":"3ad1522f-8aea-4551-af0c-131bf223b514","Type":"ContainerStarted","Data":"62019e4bb2e09c8dd9a2bd3721e41dec00a70e13fb177d513c409a1ff83b576a"}
Jul 30 19:25:18.782999 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:18.782728    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-msg4s" event={"ID":"a9c42650-7ee2-42f7-9b2d-dbf601ba95b3","Type":"ContainerStarted","Data":"905647d8ac9e0bd4908189caab448beecd0976c9b9c669813e972cba90ba02b5"}
Jul 30 19:25:18.784702 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:18.784517    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-72ngp" podStartSLOduration=2.025066992 podStartE2EDuration="10.784498879s" totalImagesPullingTime="8.759431887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:25:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:18.781627703 +0000 UTC m=+46.746208776" watchObservedRunningTime="2026-07-30 19:25:18.784498879 +0000 UTC m=+46.749079957"
Jul 30 19:25:18.785562 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:18.785526    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hqnjj" event={"ID":"cc17657c-96a1-4e8f-b659-dd8a92b29ee0","Type":"ContainerStarted","Data":"0d4c800de2d5610e423e452744b3be36833662e46c0ba2aca467e0e0343d3a7b"}
Jul 30 19:25:18.789176 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:18.789151    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-pgm6p" event={"ID":"6b2c7322-e84b-4ce7-a559-91eb3332fe03","Type":"ContainerStarted","Data":"ed46ce9002da6f3f4ad178e7a282099baab0ed5b29419b4204a6b09e3c383f78"}
Jul 30 19:25:18.791622 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:18.791596    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d" event={"ID":"a6963706-5570-4c07-a0da-707e5bc655a0","Type":"ContainerStarted","Data":"ceaed409c8252ec9b4c5b34e453a43b21ede071106f750efa7b0eadd00824d17"}
Jul 30 19:25:18.792866 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:18.792847    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d"
Jul 30 19:25:18.794128 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:18.794103    2554 patch_prober.go:28] interesting pod/console-operator-5ccd6b445f-wrp7d container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.10:8443/readyz\": dial tcp 10.132.0.10:8443: connect: connection refused" start-of-body=
Jul 30 19:25:18.794317 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:18.794285    2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d" podUID="a6963706-5570-4c07-a0da-707e5bc655a0" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.10:8443/readyz\": dial tcp 10.132.0.10:8443: connect: connection refused"
Jul 30 19:25:18.797992 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:18.795267    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-hlccf" event={"ID":"82b54e54-02dd-4fb7-81d2-97b76aa991fe","Type":"ContainerStarted","Data":"6b71a8b5d6b7795d925f830ce6ba78ca5fb745e91bff35c5893d28a1a40aec55"}
Jul 30 19:25:18.800744 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:18.800693    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-77696685b-smtsw" podStartSLOduration=1.911351787 podStartE2EDuration="10.800672753s" totalImagesPullingTime="8.889320966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:25:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:18.80001639 +0000 UTC m=+46.764597472" watchObservedRunningTime="2026-07-30 19:25:18.800672753 +0000 UTC m=+46.765253835"
Jul 30 19:25:18.858670 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:18.858602    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hqnjj" podStartSLOduration=2.037086585 podStartE2EDuration="12.858578381s" totalImagesPullingTime="10.821491796s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:25:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:18.840906715 +0000 UTC m=+46.805487790" watchObservedRunningTime="2026-07-30 19:25:18.858578381 +0000 UTC m=+46.823159462"
Jul 30 19:25:18.877018 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:18.876945    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-hlccf" podStartSLOduration=1.9565577090000001 podStartE2EDuration="10.876923896s" totalImagesPullingTime="8.920366187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:25:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:18.859040259 +0000 UTC m=+46.823621341" watchObservedRunningTime="2026-07-30 19:25:18.876923896 +0000 UTC m=+46.841504978"
Jul 30 19:25:18.878457 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:18.878236    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d" podStartSLOduration=1.9395170240000001 podStartE2EDuration="10.878220895s" totalImagesPullingTime="8.938703871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:25:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:18.876375039 +0000 UTC m=+46.840956120" watchObservedRunningTime="2026-07-30 19:25:18.878220895 +0000 UTC m=+46.842801976"
Jul 30 19:25:18.916884 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:18.916822    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-msg4s" podStartSLOduration=1.995462249 podStartE2EDuration="10.916804638s" totalImagesPullingTime="8.921342389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:25:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:18.898611378 +0000 UTC m=+46.863192461" watchObservedRunningTime="2026-07-30 19:25:18.916804638 +0000 UTC m=+46.881385718"
Jul 30 19:25:18.917378 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:18.917344    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-pgm6p" podStartSLOduration=1.800150688 podStartE2EDuration="10.917334673s" totalImagesPullingTime="9.117183985s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:25:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:18.916674244 +0000 UTC m=+46.881255326" watchObservedRunningTime="2026-07-30 19:25:18.917334673 +0000 UTC m=+46.881915786"
Jul 30 19:25:18.949766 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:18.949731    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:19.048213 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:19.048170    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:19.084439 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:19.084404    2554 patch_prober.go:28] interesting pod/console-operator-5ccd6b445f-wrp7d container/console-operator namespace/openshift-console-operator: Liveness probe status=failure output="Get \"https://10.132.0.10:8443/healthz\": dial tcp 10.132.0.10:8443: connect: connection refused" start-of-body=
Jul 30 19:25:19.084713 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:19.084679    2554 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d" podUID="a6963706-5570-4c07-a0da-707e5bc655a0" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.10:8443/healthz\": dial tcp 10.132.0.10:8443: connect: connection refused"
Jul 30 19:25:19.801054 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:19.801023    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/0.log"
Jul 30 19:25:19.801212 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:19.801071    2554 generic.go:356] "Generic (PLEG): container finished" podID="a6963706-5570-4c07-a0da-707e5bc655a0" containerID="ceaed409c8252ec9b4c5b34e453a43b21ede071106f750efa7b0eadd00824d17" exitCode=255
Jul 30 19:25:19.801212 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:19.801163    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d" event={"ID":"a6963706-5570-4c07-a0da-707e5bc655a0","Type":"ContainerDied","Data":"ceaed409c8252ec9b4c5b34e453a43b21ede071106f750efa7b0eadd00824d17"}
Jul 30 19:25:19.801441 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:19.801423    2554 scope.go:117] "RemoveContainer" containerID="ceaed409c8252ec9b4c5b34e453a43b21ede071106f750efa7b0eadd00824d17"
Jul 30 19:25:19.804388 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:19.804354    2554 generic.go:356] "Generic (PLEG): container finished" podID="3ad1522f-8aea-4551-af0c-131bf223b514" containerID="62019e4bb2e09c8dd9a2bd3721e41dec00a70e13fb177d513c409a1ff83b576a" exitCode=0
Jul 30 19:25:19.804501 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:19.804427    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5j6j7" event={"ID":"3ad1522f-8aea-4551-af0c-131bf223b514","Type":"ContainerDied","Data":"62019e4bb2e09c8dd9a2bd3721e41dec00a70e13fb177d513c409a1ff83b576a"}
Jul 30 19:25:19.868529 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:19.868455    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:19.950538 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:19.950507    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:20.004894 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.004862    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:20.138463 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.138431    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:20.141825 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.141799    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6ddl4"]
Jul 30 19:25:20.163085 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.162999    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6ddl4"]
Jul 30 19:25:20.163258 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.163147    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6ddl4"
Jul 30 19:25:20.165587 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.165562    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 30 19:25:20.165721 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.165588    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 30 19:25:20.165772 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.165729    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7gsjq\""
Jul 30 19:25:20.191585 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.191552    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:20.279697 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.279636    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngfmm\" (UniqueName: \"kubernetes.io/projected/9584099b-badd-46a7-b27d-c62012611e0a-kube-api-access-ngfmm\") pod \"insights-runtime-extractor-6ddl4\" (UID: \"9584099b-badd-46a7-b27d-c62012611e0a\") " pod="openshift-insights/insights-runtime-extractor-6ddl4"
Jul 30 19:25:20.279872 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.279778    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9584099b-badd-46a7-b27d-c62012611e0a-data-volume\") pod \"insights-runtime-extractor-6ddl4\" (UID: \"9584099b-badd-46a7-b27d-c62012611e0a\") " pod="openshift-insights/insights-runtime-extractor-6ddl4"
Jul 30 19:25:20.279910 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.279886    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9584099b-badd-46a7-b27d-c62012611e0a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6ddl4\" (UID: \"9584099b-badd-46a7-b27d-c62012611e0a\") " pod="openshift-insights/insights-runtime-extractor-6ddl4"
Jul 30 19:25:20.279995 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.279981    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9584099b-badd-46a7-b27d-c62012611e0a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6ddl4\" (UID: \"9584099b-badd-46a7-b27d-c62012611e0a\") " pod="openshift-insights/insights-runtime-extractor-6ddl4"
Jul 30 19:25:20.280039 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.280017    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9584099b-badd-46a7-b27d-c62012611e0a-crio-socket\") pod \"insights-runtime-extractor-6ddl4\" (UID: \"9584099b-badd-46a7-b27d-c62012611e0a\") " pod="openshift-insights/insights-runtime-extractor-6ddl4"
Jul 30 19:25:20.343267 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.343187    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:20.380972 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.380918    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9584099b-badd-46a7-b27d-c62012611e0a-crio-socket\") pod \"insights-runtime-extractor-6ddl4\" (UID: \"9584099b-badd-46a7-b27d-c62012611e0a\") " pod="openshift-insights/insights-runtime-extractor-6ddl4"
Jul 30 19:25:20.381145 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.381007    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngfmm\" (UniqueName: \"kubernetes.io/projected/9584099b-badd-46a7-b27d-c62012611e0a-kube-api-access-ngfmm\") pod \"insights-runtime-extractor-6ddl4\" (UID: \"9584099b-badd-46a7-b27d-c62012611e0a\") " pod="openshift-insights/insights-runtime-extractor-6ddl4"
Jul 30 19:25:20.381145 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.381092    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9584099b-badd-46a7-b27d-c62012611e0a-data-volume\") pod \"insights-runtime-extractor-6ddl4\" (UID: \"9584099b-badd-46a7-b27d-c62012611e0a\") " pod="openshift-insights/insights-runtime-extractor-6ddl4"
Jul 30 19:25:20.381145 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.381138    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9584099b-badd-46a7-b27d-c62012611e0a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6ddl4\" (UID: \"9584099b-badd-46a7-b27d-c62012611e0a\") " pod="openshift-insights/insights-runtime-extractor-6ddl4"
Jul 30 19:25:20.381255 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.381145    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9584099b-badd-46a7-b27d-c62012611e0a-crio-socket\") pod \"insights-runtime-extractor-6ddl4\" (UID: \"9584099b-badd-46a7-b27d-c62012611e0a\") " pod="openshift-insights/insights-runtime-extractor-6ddl4"
Jul 30 19:25:20.381255 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:20.381241    2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 19:25:20.381325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.381270    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9584099b-badd-46a7-b27d-c62012611e0a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6ddl4\" (UID: \"9584099b-badd-46a7-b27d-c62012611e0a\") " pod="openshift-insights/insights-runtime-extractor-6ddl4"
Jul 30 19:25:20.381325 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:20.381311    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9584099b-badd-46a7-b27d-c62012611e0a-insights-runtime-extractor-tls podName:9584099b-badd-46a7-b27d-c62012611e0a nodeName:}" failed. No retries permitted until 2026-07-30 19:25:20.881293122 +0000 UTC m=+48.845874181 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9584099b-badd-46a7-b27d-c62012611e0a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6ddl4" (UID: "9584099b-badd-46a7-b27d-c62012611e0a") : secret "insights-runtime-extractor-tls" not found
Jul 30 19:25:20.381399 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:20.381345    2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 19:25:20.381437 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:20.381404    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9584099b-badd-46a7-b27d-c62012611e0a-kube-rbac-proxy-cm podName:9584099b-badd-46a7-b27d-c62012611e0a nodeName:}" failed. No retries permitted until 2026-07-30 19:25:20.881389325 +0000 UTC m=+48.845970399 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9584099b-badd-46a7-b27d-c62012611e0a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6ddl4" (UID: "9584099b-badd-46a7-b27d-c62012611e0a") : configmap "kube-rbac-proxy" not found
Jul 30 19:25:20.381490 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.381472    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9584099b-badd-46a7-b27d-c62012611e0a-data-volume\") pod \"insights-runtime-extractor-6ddl4\" (UID: \"9584099b-badd-46a7-b27d-c62012611e0a\") " pod="openshift-insights/insights-runtime-extractor-6ddl4"
Jul 30 19:25:20.391642 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.391614    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngfmm\" (UniqueName: \"kubernetes.io/projected/9584099b-badd-46a7-b27d-c62012611e0a-kube-api-access-ngfmm\") pod \"insights-runtime-extractor-6ddl4\" (UID: \"9584099b-badd-46a7-b27d-c62012611e0a\") " pod="openshift-insights/insights-runtime-extractor-6ddl4"
Jul 30 19:25:20.445082 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.445046    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:20.773897 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.773864    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:20.809990 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.809943    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/1.log"
Jul 30 19:25:20.810365 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.810351    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/0.log"
Jul 30 19:25:20.810416 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.810384    2554 generic.go:356] "Generic (PLEG): container finished" podID="a6963706-5570-4c07-a0da-707e5bc655a0" containerID="3f0147536ebfbf26ab5bd106b3693ed99dd9af2f0ade8d7aee4471b876c2b2ef" exitCode=255
Jul 30 19:25:20.810482 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.810468    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d" event={"ID":"a6963706-5570-4c07-a0da-707e5bc655a0","Type":"ContainerDied","Data":"3f0147536ebfbf26ab5bd106b3693ed99dd9af2f0ade8d7aee4471b876c2b2ef"}
Jul 30 19:25:20.810534 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.810515    2554 scope.go:117] "RemoveContainer" containerID="ceaed409c8252ec9b4c5b34e453a43b21ede071106f750efa7b0eadd00824d17"
Jul 30 19:25:20.810780 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.810747    2554 scope.go:117] "RemoveContainer" containerID="3f0147536ebfbf26ab5bd106b3693ed99dd9af2f0ade8d7aee4471b876c2b2ef"
Jul 30 19:25:20.811006 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:20.810974    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-wrp7d_openshift-console-operator(a6963706-5570-4c07-a0da-707e5bc655a0)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d" podUID="a6963706-5570-4c07-a0da-707e5bc655a0"
Jul 30 19:25:20.813699 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.813678    2554 generic.go:356] "Generic (PLEG): container finished" podID="3ad1522f-8aea-4551-af0c-131bf223b514" containerID="bbf3e13b362f725b89fef2702dc52c219de7d884470774513e0270b9b22c89ae" exitCode=0
Jul 30 19:25:20.813781 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.813714    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5j6j7" event={"ID":"3ad1522f-8aea-4551-af0c-131bf223b514","Type":"ContainerDied","Data":"bbf3e13b362f725b89fef2702dc52c219de7d884470774513e0270b9b22c89ae"}
Jul 30 19:25:20.833635 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.833601    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:20.885532 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.885494    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9584099b-badd-46a7-b27d-c62012611e0a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6ddl4\" (UID: \"9584099b-badd-46a7-b27d-c62012611e0a\") " pod="openshift-insights/insights-runtime-extractor-6ddl4"
Jul 30 19:25:20.885690 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:20.885608    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9584099b-badd-46a7-b27d-c62012611e0a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6ddl4\" (UID: \"9584099b-badd-46a7-b27d-c62012611e0a\") " pod="openshift-insights/insights-runtime-extractor-6ddl4"
Jul 30 19:25:20.885750 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:20.885724    2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 19:25:20.885802 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:20.885791    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9584099b-badd-46a7-b27d-c62012611e0a-kube-rbac-proxy-cm podName:9584099b-badd-46a7-b27d-c62012611e0a nodeName:}" failed. No retries permitted until 2026-07-30 19:25:21.88577046 +0000 UTC m=+49.850351521 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9584099b-badd-46a7-b27d-c62012611e0a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6ddl4" (UID: "9584099b-badd-46a7-b27d-c62012611e0a") : configmap "kube-rbac-proxy" not found
Jul 30 19:25:20.885989 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:20.885948    2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 19:25:20.886094 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:20.886030    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9584099b-badd-46a7-b27d-c62012611e0a-insights-runtime-extractor-tls podName:9584099b-badd-46a7-b27d-c62012611e0a nodeName:}" failed. No retries permitted until 2026-07-30 19:25:21.886010733 +0000 UTC m=+49.850591806 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9584099b-badd-46a7-b27d-c62012611e0a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6ddl4" (UID: "9584099b-badd-46a7-b27d-c62012611e0a") : secret "insights-runtime-extractor-tls" not found
Jul 30 19:25:21.048659 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:21.048534    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:21.156041 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:21.156002    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:21.821147 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:21.821107    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5j6j7" event={"ID":"3ad1522f-8aea-4551-af0c-131bf223b514","Type":"ContainerStarted","Data":"c1a74eb4f512a2196e3c00bbfae3917cf5a3fdf0913f393708fc019f7e774310"}
Jul 30 19:25:21.822517 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:21.822495    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/1.log"
Jul 30 19:25:21.822804 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:21.822789    2554 scope.go:117] "RemoveContainer" containerID="3f0147536ebfbf26ab5bd106b3693ed99dd9af2f0ade8d7aee4471b876c2b2ef"
Jul 30 19:25:21.822948 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:21.822932    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-wrp7d_openshift-console-operator(a6963706-5570-4c07-a0da-707e5bc655a0)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d" podUID="a6963706-5570-4c07-a0da-707e5bc655a0"
Jul 30 19:25:21.843500 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:21.843429    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5j6j7" podStartSLOduration=8.863319065 podStartE2EDuration="48.843407437s" totalImagesPullingTime="39.980088372s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 19:24:33 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:21.841638656 +0000 UTC m=+49.806219738" watchObservedRunningTime="2026-07-30 19:25:21.843407437 +0000 UTC m=+49.807988519"
Jul 30 19:25:21.893322 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:21.893284    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9584099b-badd-46a7-b27d-c62012611e0a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6ddl4\" (UID: \"9584099b-badd-46a7-b27d-c62012611e0a\") " pod="openshift-insights/insights-runtime-extractor-6ddl4"
Jul 30 19:25:21.893526 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:21.893388    2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 19:25:21.893526 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:21.893450    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9584099b-badd-46a7-b27d-c62012611e0a-insights-runtime-extractor-tls podName:9584099b-badd-46a7-b27d-c62012611e0a nodeName:}" failed. No retries permitted until 2026-07-30 19:25:23.893431763 +0000 UTC m=+51.858012838 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9584099b-badd-46a7-b27d-c62012611e0a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6ddl4" (UID: "9584099b-badd-46a7-b27d-c62012611e0a") : secret "insights-runtime-extractor-tls" not found
Jul 30 19:25:21.893526 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:21.893485    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9584099b-badd-46a7-b27d-c62012611e0a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6ddl4\" (UID: \"9584099b-badd-46a7-b27d-c62012611e0a\") " pod="openshift-insights/insights-runtime-extractor-6ddl4"
Jul 30 19:25:21.894039 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:21.893996    2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 19:25:21.894183 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:21.894073    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9584099b-badd-46a7-b27d-c62012611e0a-kube-rbac-proxy-cm podName:9584099b-badd-46a7-b27d-c62012611e0a nodeName:}" failed. No retries permitted until 2026-07-30 19:25:23.894057493 +0000 UTC m=+51.858638568 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9584099b-badd-46a7-b27d-c62012611e0a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6ddl4" (UID: "9584099b-badd-46a7-b27d-c62012611e0a") : configmap "kube-rbac-proxy" not found
Jul 30 19:25:21.962318 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:21.962281    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:22.062249 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:22.062204    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:22.686263 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:22.686234    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:22.786902 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:22.786865    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:22.801566 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:22.801520    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-tls\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:22.801566 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:22.801569    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6db7b86-8d56-4418-be35-098fab4e24da-metrics-tls\") pod \"dns-default-wh4lj\" (UID: \"c6db7b86-8d56-4418-be35-098fab4e24da\") " pod="openshift-dns/dns-default-wh4lj"
Jul 30 19:25:22.801813 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:22.801670    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 19:25:22.801813 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:22.801690    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bcbd966f6-xd72m: secret "image-registry-tls" not found
Jul 30 19:25:22.801813 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:22.801732    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 19:25:22.801813 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:22.801761    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-tls podName:bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf nodeName:}" failed. No retries permitted until 2026-07-30 19:25:38.801740842 +0000 UTC m=+66.766321902 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-tls") pod "image-registry-5bcbd966f6-xd72m" (UID: "bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf") : secret "image-registry-tls" not found
Jul 30 19:25:22.801813 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:22.801790    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6db7b86-8d56-4418-be35-098fab4e24da-metrics-tls podName:c6db7b86-8d56-4418-be35-098fab4e24da nodeName:}" failed. No retries permitted until 2026-07-30 19:25:38.801778024 +0000 UTC m=+66.766359096 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c6db7b86-8d56-4418-be35-098fab4e24da-metrics-tls") pod "dns-default-wh4lj" (UID: "c6db7b86-8d56-4418-be35-098fab4e24da") : secret "dns-default-metrics-tls" not found
Jul 30 19:25:23.083074 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:23.083035    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:23.183608 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:23.183575    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:23.225832 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:23.225795    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lk9m5_5028ef75-1c24-4d52-8364-8ff8ac66f6cf/dns-node-resolver/0.log"
Jul 30 19:25:23.283666 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:23.283618    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:23.578203 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:23.578162    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:23.910164 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:23.910078    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9584099b-badd-46a7-b27d-c62012611e0a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6ddl4\" (UID: \"9584099b-badd-46a7-b27d-c62012611e0a\") " pod="openshift-insights/insights-runtime-extractor-6ddl4"
Jul 30 19:25:23.910164 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:23.910129    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9584099b-badd-46a7-b27d-c62012611e0a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6ddl4\" (UID: \"9584099b-badd-46a7-b27d-c62012611e0a\") " pod="openshift-insights/insights-runtime-extractor-6ddl4"
Jul 30 19:25:23.910552 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:23.910237    2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 19:25:23.910552 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:23.910307    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9584099b-badd-46a7-b27d-c62012611e0a-insights-runtime-extractor-tls podName:9584099b-badd-46a7-b27d-c62012611e0a nodeName:}" failed. No retries permitted until 2026-07-30 19:25:27.910289928 +0000 UTC m=+55.874871000 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9584099b-badd-46a7-b27d-c62012611e0a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6ddl4" (UID: "9584099b-badd-46a7-b27d-c62012611e0a") : secret "insights-runtime-extractor-tls" not found
Jul 30 19:25:23.910622 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:23.910571    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9584099b-badd-46a7-b27d-c62012611e0a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6ddl4\" (UID: \"9584099b-badd-46a7-b27d-c62012611e0a\") " pod="openshift-insights/insights-runtime-extractor-6ddl4"
Jul 30 19:25:24.024864 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:24.024838    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cdmwn_5bdbf9db-4a68-4625-adc1-a17b0e04d581/node-ca/0.log"
Jul 30 19:25:24.717976 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:24.717917    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/749026e1-d4ac-408c-b989-e79e0e95306a-service-ca-bundle\") pod \"router-default-54bd4f7fb-wwb4d\" (UID: \"749026e1-d4ac-408c-b989-e79e0e95306a\") " pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:24.718222 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:24.718110    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/749026e1-d4ac-408c-b989-e79e0e95306a-metrics-certs\") pod \"router-default-54bd4f7fb-wwb4d\" (UID: \"749026e1-d4ac-408c-b989-e79e0e95306a\") " pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:24.718222 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:24.718161    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/749026e1-d4ac-408c-b989-e79e0e95306a-service-ca-bundle podName:749026e1-d4ac-408c-b989-e79e0e95306a nodeName:}" failed. No retries permitted until 2026-07-30 19:25:40.718137733 +0000 UTC m=+68.682718798 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/749026e1-d4ac-408c-b989-e79e0e95306a-service-ca-bundle") pod "router-default-54bd4f7fb-wwb4d" (UID: "749026e1-d4ac-408c-b989-e79e0e95306a") : configmap references non-existent config key: service-ca.crt
Jul 30 19:25:24.718222 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:24.718201    2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 19:25:24.718329 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:24.718245    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/749026e1-d4ac-408c-b989-e79e0e95306a-metrics-certs podName:749026e1-d4ac-408c-b989-e79e0e95306a nodeName:}" failed. No retries permitted until 2026-07-30 19:25:40.718234764 +0000 UTC m=+68.682815822 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/749026e1-d4ac-408c-b989-e79e0e95306a-metrics-certs") pod "router-default-54bd4f7fb-wwb4d" (UID: "749026e1-d4ac-408c-b989-e79e0e95306a") : secret "router-metrics-certs-default" not found
Jul 30 19:25:24.920180 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:24.920137    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ea2bd76b-48c7-417f-ab1e-a1f1414c0aab-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-9h8w7\" (UID: \"ea2bd76b-48c7-417f-ab1e-a1f1414c0aab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9h8w7"
Jul 30 19:25:24.920564 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:24.920216    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efcb8db8-2128-45ea-9098-ddf248716241-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-ft7tw\" (UID: \"efcb8db8-2128-45ea-9098-ddf248716241\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-ft7tw"
Jul 30 19:25:24.920564 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:24.920248    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/53668666-0a1b-402a-9c46-dd0835d55256-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-flnc7\" (UID: \"53668666-0a1b-402a-9c46-dd0835d55256\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-flnc7"
Jul 30 19:25:24.920564 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:24.920303    2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 30 19:25:24.920564 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:24.920353    2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 30 19:25:24.920564 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:24.920356    2554 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 19:25:24.920564 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:24.920367    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea2bd76b-48c7-417f-ab1e-a1f1414c0aab-samples-operator-tls podName:ea2bd76b-48c7-417f-ab1e-a1f1414c0aab nodeName:}" failed. No retries permitted until 2026-07-30 19:25:40.920347725 +0000 UTC m=+68.884928808 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ea2bd76b-48c7-417f-ab1e-a1f1414c0aab-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-9h8w7" (UID: "ea2bd76b-48c7-417f-ab1e-a1f1414c0aab") : secret "samples-operator-tls" not found
Jul 30 19:25:24.920564 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:24.920433    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53668666-0a1b-402a-9c46-dd0835d55256-cluster-monitoring-operator-tls podName:53668666-0a1b-402a-9c46-dd0835d55256 nodeName:}" failed. No retries permitted until 2026-07-30 19:25:40.920415669 +0000 UTC m=+68.884996731 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/53668666-0a1b-402a-9c46-dd0835d55256-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-flnc7" (UID: "53668666-0a1b-402a-9c46-dd0835d55256") : secret "cluster-monitoring-operator-tls" not found
Jul 30 19:25:24.920564 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:24.920499    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efcb8db8-2128-45ea-9098-ddf248716241-networking-console-plugin-cert podName:efcb8db8-2128-45ea-9098-ddf248716241 nodeName:}" failed. No retries permitted until 2026-07-30 19:25:40.92049044 +0000 UTC m=+68.885071499 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/efcb8db8-2128-45ea-9098-ddf248716241-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-ft7tw" (UID: "efcb8db8-2128-45ea-9098-ddf248716241") : secret "networking-console-plugin-cert" not found
Jul 30 19:25:25.025517 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:25.025428    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hqnjj_cc17657c-96a1-4e8f-b659-dd8a92b29ee0/serve-healthcheck-canary/0.log"
Jul 30 19:25:25.626261 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:25.626223    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-72ngp_083469f7-657e-4dc8-98ac-70c9749d5922/kube-storage-version-migrator-operator/0.log"
Jul 30 19:25:25.756206 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:25.756176    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:26.384317 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:26.384281    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:26.432001 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:26.431945    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:27.948144 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:27.948098    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9584099b-badd-46a7-b27d-c62012611e0a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6ddl4\" (UID: \"9584099b-badd-46a7-b27d-c62012611e0a\") " pod="openshift-insights/insights-runtime-extractor-6ddl4"
Jul 30 19:25:27.948537 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:27.948260    2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 19:25:27.948537 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:27.948339    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9584099b-badd-46a7-b27d-c62012611e0a-insights-runtime-extractor-tls podName:9584099b-badd-46a7-b27d-c62012611e0a nodeName:}" failed. No retries permitted until 2026-07-30 19:25:35.948322651 +0000 UTC m=+63.912903710 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9584099b-badd-46a7-b27d-c62012611e0a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6ddl4" (UID: "9584099b-badd-46a7-b27d-c62012611e0a") : secret "insights-runtime-extractor-tls" not found
Jul 30 19:25:28.068933 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:28.068899    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:28.170448 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:28.170401    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:28.270176 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:28.270136    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:28.511178 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:28.511144    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:28.612461 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:28.612374    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:28.712046 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:28.712015    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:28.792856 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:28.792814    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d"
Jul 30 19:25:28.793266 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:28.793251    2554 scope.go:117] "RemoveContainer" containerID="3f0147536ebfbf26ab5bd106b3693ed99dd9af2f0ade8d7aee4471b876c2b2ef"
Jul 30 19:25:28.793429 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:28.793411    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-wrp7d_openshift-console-operator(a6963706-5570-4c07-a0da-707e5bc655a0)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d" podUID="a6963706-5570-4c07-a0da-707e5bc655a0"
Jul 30 19:25:28.812060 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:28.812028    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:29.084783 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:29.084748    2554 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d"
Jul 30 19:25:29.085183 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:29.085131    2554 scope.go:117] "RemoveContainer" containerID="3f0147536ebfbf26ab5bd106b3693ed99dd9af2f0ade8d7aee4471b876c2b2ef"
Jul 30 19:25:29.085302 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:25:29.085282    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-wrp7d_openshift-console-operator(a6963706-5570-4c07-a0da-707e5bc655a0)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d" podUID="a6963706-5570-4c07-a0da-707e5bc655a0"
Jul 30 19:25:29.581639 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:29.581596    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:30.093125 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:30.093094    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:30.193824 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:30.193788    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:30.376920 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:30.376816    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:30.770918 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:30.770886    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:30.971338 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:30.971306    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:31.364141 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:31.364107    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:31.766406 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:31.766378    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:31.879397 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:31.879364    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:32.170334 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:32.170253    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:32.567759 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:32.567712    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:32.698078 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:32.698048    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qnhsf"
Jul 30 19:25:32.774502 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:32.774469    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:32.823919 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:32.823837    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:32.968091 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:32.968057    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:33.371485 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:33.371453    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:33.971361 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:33.971329    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:34.356171 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:34.356093    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:34.568822 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:34.568790    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:34.778550 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:34.778514    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:34.970888 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:34.970846    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:35.365069 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:35.365029    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:36.023022 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:36.022988    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9584099b-badd-46a7-b27d-c62012611e0a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6ddl4\" (UID: \"9584099b-badd-46a7-b27d-c62012611e0a\") " pod="openshift-insights/insights-runtime-extractor-6ddl4"
Jul 30 19:25:36.025240 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:36.025215    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9584099b-badd-46a7-b27d-c62012611e0a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6ddl4\" (UID: \"9584099b-badd-46a7-b27d-c62012611e0a\") " pod="openshift-insights/insights-runtime-extractor-6ddl4"
Jul 30 19:25:36.075925 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:36.075888    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7gsjq\""
Jul 30 19:25:36.083568 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:36.083544    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6ddl4"
Jul 30 19:25:36.164824 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:36.164780    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:36.210316 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:36.210283    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6ddl4"]
Jul 30 19:25:36.213305 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:25:36.213275    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9584099b_badd_46a7_b27d_c62012611e0a.slice/crio-03c4bcd2fe4e1d734d43ccee01acef199d77373c3a05c472347ce01ae809a53a WatchSource:0}: Error finding container 03c4bcd2fe4e1d734d43ccee01acef199d77373c3a05c472347ce01ae809a53a: Status 404 returned error can't find the container with id 03c4bcd2fe4e1d734d43ccee01acef199d77373c3a05c472347ce01ae809a53a
Jul 30 19:25:36.373597 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:36.373501    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:36.770876 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:36.770840    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:36.874453 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:36.874403    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6ddl4" event={"ID":"9584099b-badd-46a7-b27d-c62012611e0a","Type":"ContainerStarted","Data":"0cbca4c7870715b579a8e437a81dbad47b089eda61b2b532a3f2a598edc4c56e"}
Jul 30 19:25:36.874613 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:36.874480    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6ddl4" event={"ID":"9584099b-badd-46a7-b27d-c62012611e0a","Type":"ContainerStarted","Data":"03c4bcd2fe4e1d734d43ccee01acef199d77373c3a05c472347ce01ae809a53a"}
Jul 30 19:25:36.968914 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:36.968878    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:37.133337 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:37.133254    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv49g\" (UniqueName: \"kubernetes.io/projected/f218f763-1cba-4443-9784-ba8b9452bb6c-kube-api-access-mv49g\") pod \"network-check-target-8nv66\" (UID: \"f218f763-1cba-4443-9784-ba8b9452bb6c\") " pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:25:37.133337 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:37.133331    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aec35b15-c096-4e99-a7dc-77be12a9ceca-metrics-certs\") pod \"network-metrics-daemon-dtpk6\" (UID: \"aec35b15-c096-4e99-a7dc-77be12a9ceca\") " pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:25:37.135508 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:37.135481    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 19:25:37.135868 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:37.135849    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv49g\" (UniqueName: \"kubernetes.io/projected/f218f763-1cba-4443-9784-ba8b9452bb6c-kube-api-access-mv49g\") pod \"network-check-target-8nv66\" (UID: \"f218f763-1cba-4443-9784-ba8b9452bb6c\") " pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:25:37.146647 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:37.146621    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aec35b15-c096-4e99-a7dc-77be12a9ceca-metrics-certs\") pod \"network-metrics-daemon-dtpk6\" (UID: \"aec35b15-c096-4e99-a7dc-77be12a9ceca\") " pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:25:37.195445 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:37.195413    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-82x98\""
Jul 30 19:25:37.203264 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:37.203242    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:25:37.234854 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:37.234825    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c766173-91ba-4e51-923b-51ec626a0bee-original-pull-secret\") pod \"global-pull-secret-syncer-cjhlf\" (UID: \"6c766173-91ba-4e51-923b-51ec626a0bee\") " pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:25:37.237191 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:37.237167    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 19:25:37.248377 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:37.248335    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6c766173-91ba-4e51-923b-51ec626a0bee-original-pull-secret\") pod \"global-pull-secret-syncer-cjhlf\" (UID: \"6c766173-91ba-4e51-923b-51ec626a0bee\") " pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:25:37.293610 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:37.293582    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bgfkv\""
Jul 30 19:25:37.296450 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:37.296424    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cjhlf"
Jul 30 19:25:37.302188 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:37.302163    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dtpk6"
Jul 30 19:25:37.347645 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:37.347617    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8nv66"]
Jul 30 19:25:37.351586 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:25:37.351553    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf218f763_1cba_4443_9784_ba8b9452bb6c.slice/crio-8c9cc11f9f93cd85a1c345abc424dd921a47ac534014f2f86dd1e56d5b1a7ad5 WatchSource:0}: Error finding container 8c9cc11f9f93cd85a1c345abc424dd921a47ac534014f2f86dd1e56d5b1a7ad5: Status 404 returned error can't find the container with id 8c9cc11f9f93cd85a1c345abc424dd921a47ac534014f2f86dd1e56d5b1a7ad5
Jul 30 19:25:37.451193 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:37.451157    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cjhlf"]
Jul 30 19:25:37.454170 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:25:37.454141    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c766173_91ba_4e51_923b_51ec626a0bee.slice/crio-068752c6308bf6112c62fb66738659cf470e9131d852f97e43c3277f2c154ca1 WatchSource:0}: Error finding container 068752c6308bf6112c62fb66738659cf470e9131d852f97e43c3277f2c154ca1: Status 404 returned error can't find the container with id 068752c6308bf6112c62fb66738659cf470e9131d852f97e43c3277f2c154ca1
Jul 30 19:25:37.471013 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:37.470987    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dtpk6"]
Jul 30 19:25:37.473981 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:25:37.473939    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaec35b15_c096_4e99_a7dc_77be12a9ceca.slice/crio-778e367cf444e56f8a213298f217bd13023d1d802b56c280c7e3a8463a6855f0 WatchSource:0}: Error finding container 778e367cf444e56f8a213298f217bd13023d1d802b56c280c7e3a8463a6855f0: Status 404 returned error can't find the container with id 778e367cf444e56f8a213298f217bd13023d1d802b56c280c7e3a8463a6855f0
Jul 30 19:25:37.570074 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:37.570011    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:37.880487 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:37.880446    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dtpk6" event={"ID":"aec35b15-c096-4e99-a7dc-77be12a9ceca","Type":"ContainerStarted","Data":"778e367cf444e56f8a213298f217bd13023d1d802b56c280c7e3a8463a6855f0"}
Jul 30 19:25:37.882010 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:37.881981    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cjhlf" event={"ID":"6c766173-91ba-4e51-923b-51ec626a0bee","Type":"ContainerStarted","Data":"068752c6308bf6112c62fb66738659cf470e9131d852f97e43c3277f2c154ca1"}
Jul 30 19:25:37.883717 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:37.883690    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8nv66" event={"ID":"f218f763-1cba-4443-9784-ba8b9452bb6c","Type":"ContainerStarted","Data":"a1381ca9e6db8428cc99b7f0f52c12d9fa05a57a51e8b803e80c41a294504ba5"}
Jul 30 19:25:37.883861 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:37.883751    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8nv66" event={"ID":"f218f763-1cba-4443-9784-ba8b9452bb6c","Type":"ContainerStarted","Data":"8c9cc11f9f93cd85a1c345abc424dd921a47ac534014f2f86dd1e56d5b1a7ad5"}
Jul 30 19:25:37.883861 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:37.883781    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:25:37.885894 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:37.885867    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6ddl4" event={"ID":"9584099b-badd-46a7-b27d-c62012611e0a","Type":"ContainerStarted","Data":"7decabc73139c85697e4e90442c38da8eadd612fb2a7b1920d65c5d8d07a0f51"}
Jul 30 19:25:37.899626 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:37.899573    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8nv66" podStartSLOduration=64.899552895 podStartE2EDuration="1m4.899552895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:24:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:37.898125982 +0000 UTC m=+65.862707064" watchObservedRunningTime="2026-07-30 19:25:37.899552895 +0000 UTC m=+65.864133955"
Jul 30 19:25:37.969907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:37.969871    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:38.371355 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:38.371319    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:38.567408 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:38.567372    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:38.770471 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:38.770437    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:38.851776 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:38.851731    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6db7b86-8d56-4418-be35-098fab4e24da-metrics-tls\") pod \"dns-default-wh4lj\" (UID: \"c6db7b86-8d56-4418-be35-098fab4e24da\") " pod="openshift-dns/dns-default-wh4lj"
Jul 30 19:25:38.852080 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:38.852056    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-tls\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:38.854668 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:38.854626    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6db7b86-8d56-4418-be35-098fab4e24da-metrics-tls\") pod \"dns-default-wh4lj\" (UID: \"c6db7b86-8d56-4418-be35-098fab4e24da\") " pod="openshift-dns/dns-default-wh4lj"
Jul 30 19:25:38.854848 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:38.854825    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-tls\") pod \"image-registry-5bcbd966f6-xd72m\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") " pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:39.019220 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:39.019171    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9rs47\""
Jul 30 19:25:39.028646 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:39.028215    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:39.031530 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:39.031157    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9htd7\""
Jul 30 19:25:39.039142 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:39.039099    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wh4lj"
Jul 30 19:25:39.206128 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:39.206075    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bcbd966f6-xd72m"]
Jul 30 19:25:39.208460 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:25:39.208432    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdbedcab_2bf0_4b28_abe0_9ccbeadfddcf.slice/crio-d73ffc9e82a6f42671522b1e40a6515fe2b22220ea219c0196531c81e91fa6ab WatchSource:0}: Error finding container d73ffc9e82a6f42671522b1e40a6515fe2b22220ea219c0196531c81e91fa6ab: Status 404 returned error can't find the container with id d73ffc9e82a6f42671522b1e40a6515fe2b22220ea219c0196531c81e91fa6ab
Jul 30 19:25:39.218464 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:39.218428    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wh4lj"]
Jul 30 19:25:39.221429 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:25:39.221399    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6db7b86_8d56_4418_be35_098fab4e24da.slice/crio-e67e4ca8a76e74af0d64f5bfb767cfe55d12100d43f295a055b9e24cb14e56f0 WatchSource:0}: Error finding container e67e4ca8a76e74af0d64f5bfb767cfe55d12100d43f295a055b9e24cb14e56f0: Status 404 returned error can't find the container with id e67e4ca8a76e74af0d64f5bfb767cfe55d12100d43f295a055b9e24cb14e56f0
Jul 30 19:25:39.896723 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:39.896685    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6ddl4" event={"ID":"9584099b-badd-46a7-b27d-c62012611e0a","Type":"ContainerStarted","Data":"7e61eef7ebdf4452d3bd7f83a9c0073be31978bebe243944c6f46f44cd29c7de"}
Jul 30 19:25:39.898474 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:39.898447    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dtpk6" event={"ID":"aec35b15-c096-4e99-a7dc-77be12a9ceca","Type":"ContainerStarted","Data":"e0f11f5973431a933bce44d73754d9f19c50e5a85311632c280337dfb3b78bda"}
Jul 30 19:25:39.898585 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:39.898501    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dtpk6" event={"ID":"aec35b15-c096-4e99-a7dc-77be12a9ceca","Type":"ContainerStarted","Data":"ac845dfe016de3d819579a181aba5a0f98845e19312f7e7681c617b5a36912f8"}
Jul 30 19:25:39.899567 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:39.899547    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wh4lj" event={"ID":"c6db7b86-8d56-4418-be35-098fab4e24da","Type":"ContainerStarted","Data":"e67e4ca8a76e74af0d64f5bfb767cfe55d12100d43f295a055b9e24cb14e56f0"}
Jul 30 19:25:39.900994 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:39.900968    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m" event={"ID":"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf","Type":"ContainerStarted","Data":"b82e67e08f73a258195be6620bd617068cd4f51f911a00074ed4824735141ee2"}
Jul 30 19:25:39.901112 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:39.901008    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m" event={"ID":"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf","Type":"ContainerStarted","Data":"d73ffc9e82a6f42671522b1e40a6515fe2b22220ea219c0196531c81e91fa6ab"}
Jul 30 19:25:39.901112 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:39.901078    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:25:39.915210 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:39.915160    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6ddl4" podStartSLOduration=17.32286808 podStartE2EDuration="19.915141276s" totalImagesPullingTime="2.592273196s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 19:25:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:39.914026332 +0000 UTC m=+67.878607413" watchObservedRunningTime="2026-07-30 19:25:39.915141276 +0000 UTC m=+67.879722360"
Jul 30 19:25:39.930289 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:39.930238    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dtpk6" podStartSLOduration=65.488779864 podStartE2EDuration="1m6.930223265s" totalImagesPullingTime="1.441443401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:24:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:39.928596269 +0000 UTC m=+67.893177352" watchObservedRunningTime="2026-07-30 19:25:39.930223265 +0000 UTC m=+67.894804346"
Jul 30 19:25:39.950307 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:39.950237    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m" podStartSLOduration=66.950212254 podStartE2EDuration="1m6.950212254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:24:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:39.948818609 +0000 UTC m=+67.913399690" watchObservedRunningTime="2026-07-30 19:25:39.950212254 +0000 UTC m=+67.914793336"
Jul 30 19:25:40.481657 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:40.481620    2554 scope.go:117] "RemoveContainer" containerID="3f0147536ebfbf26ab5bd106b3693ed99dd9af2f0ade8d7aee4471b876c2b2ef"
Jul 30 19:25:40.772587 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:40.772480    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/749026e1-d4ac-408c-b989-e79e0e95306a-service-ca-bundle\") pod \"router-default-54bd4f7fb-wwb4d\" (UID: \"749026e1-d4ac-408c-b989-e79e0e95306a\") " pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:40.772587 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:40.772543    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/749026e1-d4ac-408c-b989-e79e0e95306a-metrics-certs\") pod \"router-default-54bd4f7fb-wwb4d\" (UID: \"749026e1-d4ac-408c-b989-e79e0e95306a\") " pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:40.773257 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:40.773227    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/749026e1-d4ac-408c-b989-e79e0e95306a-service-ca-bundle\") pod \"router-default-54bd4f7fb-wwb4d\" (UID: \"749026e1-d4ac-408c-b989-e79e0e95306a\") " pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:40.774844 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:40.774823    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/749026e1-d4ac-408c-b989-e79e0e95306a-metrics-certs\") pod \"router-default-54bd4f7fb-wwb4d\" (UID: \"749026e1-d4ac-408c-b989-e79e0e95306a\") " pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:40.906986 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:40.906940    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/1.log"
Jul 30 19:25:40.907411 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:40.907089    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d" event={"ID":"a6963706-5570-4c07-a0da-707e5bc655a0","Type":"ContainerStarted","Data":"f6b8bda5f5e0f1fdc3846763399510de3586b6cb2233a9e4462c18275153835e"}
Jul 30 19:25:40.907905 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:40.907880    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d"
Jul 30 19:25:40.964633 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:40.964598    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-8pxkx\""
Jul 30 19:25:40.972653 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:40.972617    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:40.974854 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:40.974176    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efcb8db8-2128-45ea-9098-ddf248716241-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-ft7tw\" (UID: \"efcb8db8-2128-45ea-9098-ddf248716241\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-ft7tw"
Jul 30 19:25:40.974854 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:40.974309    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/53668666-0a1b-402a-9c46-dd0835d55256-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-flnc7\" (UID: \"53668666-0a1b-402a-9c46-dd0835d55256\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-flnc7"
Jul 30 19:25:40.974854 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:40.974497    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ea2bd76b-48c7-417f-ab1e-a1f1414c0aab-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-9h8w7\" (UID: \"ea2bd76b-48c7-417f-ab1e-a1f1414c0aab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9h8w7"
Jul 30 19:25:40.977612 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:40.977588    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/53668666-0a1b-402a-9c46-dd0835d55256-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-flnc7\" (UID: \"53668666-0a1b-402a-9c46-dd0835d55256\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-flnc7"
Jul 30 19:25:40.977702 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:40.977588    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efcb8db8-2128-45ea-9098-ddf248716241-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-ft7tw\" (UID: \"efcb8db8-2128-45ea-9098-ddf248716241\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-ft7tw"
Jul 30 19:25:40.977702 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:40.977637    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ea2bd76b-48c7-417f-ab1e-a1f1414c0aab-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-9h8w7\" (UID: \"ea2bd76b-48c7-417f-ab1e-a1f1414c0aab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9h8w7"
Jul 30 19:25:41.099709 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:41.099615    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-fm2mn\""
Jul 30 19:25:41.107629 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:41.107596    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9h8w7"
Jul 30 19:25:41.123142 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:41.123115    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-cgq8c\""
Jul 30 19:25:41.131913 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:41.131887    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-flnc7"
Jul 30 19:25:41.147114 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:41.147084    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-b7hvf\""
Jul 30 19:25:41.155595 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:41.155570    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-ft7tw"
Jul 30 19:25:41.183591 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:41.183564    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-5ccd6b445f-wrp7d"
Jul 30 19:25:41.357457 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:41.357354    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:41.457763 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:41.457703    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:41.509372 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:41.509340    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:41.655187 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:41.655097    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:42.035768 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:42.035728    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:42.136269 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:42.136219    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:42.442058 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:42.441975    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:42.853268 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:42.853093    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:42.912633 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:42.912592    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:42.966428 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:42.966351    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:43.022803 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:43.022726    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-ft7tw"]
Jul 30 19:25:43.032531 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:25:43.032499    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefcb8db8_2128_45ea_9098_ddf248716241.slice/crio-a3b2318153f26a1c56accf466d427366a8b42d327da0184563ba26a692dc0005 WatchSource:0}: Error finding container a3b2318153f26a1c56accf466d427366a8b42d327da0184563ba26a692dc0005: Status 404 returned error can't find the container with id a3b2318153f26a1c56accf466d427366a8b42d327da0184563ba26a692dc0005
Jul 30 19:25:43.062888 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:43.062855    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:43.260615 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:43.260586    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-flnc7"]
Jul 30 19:25:43.263908 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:25:43.263868    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53668666_0a1b_402a_9c46_dd0835d55256.slice/crio-e789cdb4a05a09c4cf5c984c1d050d1f68fea966b91b43fba0fd52daed2eefac WatchSource:0}: Error finding container e789cdb4a05a09c4cf5c984c1d050d1f68fea966b91b43fba0fd52daed2eefac: Status 404 returned error can't find the container with id e789cdb4a05a09c4cf5c984c1d050d1f68fea966b91b43fba0fd52daed2eefac
Jul 30 19:25:43.265545 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:43.265519    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9h8w7"]
Jul 30 19:25:43.268637 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:43.268615    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-54bd4f7fb-wwb4d"]
Jul 30 19:25:43.272786 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:25:43.272761    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod749026e1_d4ac_408c_b989_e79e0e95306a.slice/crio-84cc5fd69e7d1a9bddf782fa0187d0b52a7ce970664ac6016cdbbdcc2f967843 WatchSource:0}: Error finding container 84cc5fd69e7d1a9bddf782fa0187d0b52a7ce970664ac6016cdbbdcc2f967843: Status 404 returned error can't find the container with id 84cc5fd69e7d1a9bddf782fa0187d0b52a7ce970664ac6016cdbbdcc2f967843
Jul 30 19:25:43.280844 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:43.280815    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:43.330726 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:43.330688    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:43.928001 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:43.927945    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wh4lj" event={"ID":"c6db7b86-8d56-4418-be35-098fab4e24da","Type":"ContainerStarted","Data":"7beff09aad1af63321f8a625731486af10e02f4dc88017e7f69387053c811566"}
Jul 30 19:25:43.928202 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:43.928028    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wh4lj" event={"ID":"c6db7b86-8d56-4418-be35-098fab4e24da","Type":"ContainerStarted","Data":"c5ffb986b09ba78e2e9373fd1d7ccf6a8fb2ccde72515f3e6d5d4b17f02533cb"}
Jul 30 19:25:43.928265 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:43.928233    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wh4lj"
Jul 30 19:25:43.929552 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:43.929488    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9h8w7" event={"ID":"ea2bd76b-48c7-417f-ab1e-a1f1414c0aab","Type":"ContainerStarted","Data":"da4984a5f66d9361f91e10e008379c6e68416a1e9741080c9390b2e1ccee064c"}
Jul 30 19:25:43.931044 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:43.931017    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-ft7tw" event={"ID":"efcb8db8-2128-45ea-9098-ddf248716241","Type":"ContainerStarted","Data":"a3b2318153f26a1c56accf466d427366a8b42d327da0184563ba26a692dc0005"}
Jul 30 19:25:43.934079 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:43.934055    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-54bd4f7fb-wwb4d" event={"ID":"749026e1-d4ac-408c-b989-e79e0e95306a","Type":"ContainerStarted","Data":"5336bc0d607fd3714d758d3ece5eec723590bb824dbbf527f6de6a88edf42dc9"}
Jul 30 19:25:43.934179 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:43.934108    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-54bd4f7fb-wwb4d" event={"ID":"749026e1-d4ac-408c-b989-e79e0e95306a","Type":"ContainerStarted","Data":"84cc5fd69e7d1a9bddf782fa0187d0b52a7ce970664ac6016cdbbdcc2f967843"}
Jul 30 19:25:43.935528 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:43.935504    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-flnc7" event={"ID":"53668666-0a1b-402a-9c46-dd0835d55256","Type":"ContainerStarted","Data":"e789cdb4a05a09c4cf5c984c1d050d1f68fea966b91b43fba0fd52daed2eefac"}
Jul 30 19:25:43.937252 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:43.937229    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cjhlf" event={"ID":"6c766173-91ba-4e51-923b-51ec626a0bee","Type":"ContainerStarted","Data":"50f9da11d0df9404fa47c56091c7e20c5e99162ddfe70763712d7c8b7a16bd75"}
Jul 30 19:25:43.947238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:43.946745    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wh4lj" podStartSLOduration=34.362487269 podStartE2EDuration="37.946723389s" totalImagesPullingTime="3.58423612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:25:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:43.945088874 +0000 UTC m=+71.909669956" watchObservedRunningTime="2026-07-30 19:25:43.946723389 +0000 UTC m=+71.911304474"
Jul 30 19:25:43.973894 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:43.973799    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:43.977022 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:43.976991    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:43.978165 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:43.978123    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-cjhlf" podStartSLOduration=65.628316641 podStartE2EDuration="1m10.978108149s" totalImagesPullingTime="5.349791508s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 19:24:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:43.961437929 +0000 UTC m=+71.926019013" watchObservedRunningTime="2026-07-30 19:25:43.978108149 +0000 UTC m=+71.942689227"
Jul 30 19:25:43.978898 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:43.978854    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-54bd4f7fb-wwb4d" podStartSLOduration=35.978842915 podStartE2EDuration="35.978842915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:25:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:43.97768173 +0000 UTC m=+71.942262806" watchObservedRunningTime="2026-07-30 19:25:43.978842915 +0000 UTC m=+71.943423997"
Jul 30 19:25:44.943531 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:44.943488    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-ft7tw" event={"ID":"efcb8db8-2128-45ea-9098-ddf248716241","Type":"ContainerStarted","Data":"9f847f4e949e3dfb2b283e654a542c10d8cc32f6e527560535bf663a0aa780dd"}
Jul 30 19:25:44.944191 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:44.944164    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:44.945551 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:44.945529    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-54bd4f7fb-wwb4d"
Jul 30 19:25:44.961518 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:44.961440    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-f77f48c6d-ft7tw" podStartSLOduration=35.836260171 podStartE2EDuration="36.961420236s" totalImagesPullingTime="1.125160065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:25:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:44.961088554 +0000 UTC m=+72.925669636" watchObservedRunningTime="2026-07-30 19:25:44.961420236 +0000 UTC m=+72.926001317"
Jul 30 19:25:45.087207 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:45.087113    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:45.151104 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:45.151064    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:45.216720 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:45.216673    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:45.319542 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:45.319506    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:45.416982 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:45.416927    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:45.471397 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:45.471319    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:46.764038 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:46.764002    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:46.819197 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:46.819163    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:46.880252 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:46.880157    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:46.946575 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:46.946546    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:46.951269 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:46.951223    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9h8w7" event={"ID":"ea2bd76b-48c7-417f-ab1e-a1f1414c0aab","Type":"ContainerStarted","Data":"111626f13b7d2b324957b461675d8f58fefbf71dc4a0c4749ff0a9894bf0ae39"}
Jul 30 19:25:46.951458 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:46.951287    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9h8w7" event={"ID":"ea2bd76b-48c7-417f-ab1e-a1f1414c0aab","Type":"ContainerStarted","Data":"1110479451b7e77ba070ed1220e260b71658d2bacdec5a999c940c8251c6c338"}
Jul 30 19:25:46.952810 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:46.952784    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-flnc7" event={"ID":"53668666-0a1b-402a-9c46-dd0835d55256","Type":"ContainerStarted","Data":"d3b2098c6ed43f046a3f85d5e0dce46b21f96e0a63344a17059e8d5a3bc086cd"}
Jul 30 19:25:46.968201 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:46.968147    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-9h8w7" podStartSLOduration=35.932324448 podStartE2EDuration="38.968129226s" totalImagesPullingTime="3.035804778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:25:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:46.96685595 +0000 UTC m=+74.931437034" watchObservedRunningTime="2026-07-30 19:25:46.968129226 +0000 UTC m=+74.932710306"
Jul 30 19:25:46.986212 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:46.986148    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-flnc7" podStartSLOduration=35.914535568 podStartE2EDuration="38.986126333s" totalImagesPullingTime="3.071590765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:25:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:25:46.98610949 +0000 UTC m=+74.950690572" watchObservedRunningTime="2026-07-30 19:25:46.986126333 +0000 UTC m=+74.950707415"
Jul 30 19:25:47.263034 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:47.262996    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:51.233683 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:51.233646    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:51.285842 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:51.285813    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:53.946232 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:53.946191    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wh4lj"
Jul 30 19:25:55.246947 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:55.246802    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:55.307444 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:55.307412    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:55.940677 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:55.940642    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:55.999256 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:55.999219    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:56.068900 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:56.068854    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:59.032473 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:59.032439    2554 patch_prober.go:28] interesting pod/image-registry-5bcbd966f6-xd72m container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 30 19:25:59.032905 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:59.032491    2554 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m" podUID="bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 19:25:59.666364 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:59.666331    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:25:59.757362 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:25:59.757331    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:00.869090 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:00.869057    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-9d65d6646-hc5j5"]
Jul 30 19:26:00.871936 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:00.871917    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:00.874337 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:00.874315    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 30 19:26:00.874784 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:00.874764    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-smf57\""
Jul 30 19:26:00.875474 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:00.875457    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 30 19:26:00.875631 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:00.875555    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 30 19:26:00.875631 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:00.875580    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 30 19:26:00.875631 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:00.875604    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 30 19:26:00.875854 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:00.875648    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 30 19:26:00.875854 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:00.875681    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 30 19:26:00.880734 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:00.880714    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 30 19:26:00.882875 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:00.882854    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9d65d6646-hc5j5"]
Jul 30 19:26:00.913114 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:00.913085    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:26:01.032022 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:01.031990    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aa052cb1-c5e2-465a-89f2-a53c960e51fb-oauth-serving-cert\") pod \"console-9d65d6646-hc5j5\" (UID: \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\") " pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:01.032189 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:01.032034    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aa052cb1-c5e2-465a-89f2-a53c960e51fb-console-config\") pod \"console-9d65d6646-hc5j5\" (UID: \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\") " pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:01.032189 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:01.032157    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nhcv\" (UniqueName: \"kubernetes.io/projected/aa052cb1-c5e2-465a-89f2-a53c960e51fb-kube-api-access-6nhcv\") pod \"console-9d65d6646-hc5j5\" (UID: \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\") " pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:01.032296 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:01.032261    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa052cb1-c5e2-465a-89f2-a53c960e51fb-trusted-ca-bundle\") pod \"console-9d65d6646-hc5j5\" (UID: \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\") " pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:01.032351 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:01.032303    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aa052cb1-c5e2-465a-89f2-a53c960e51fb-console-serving-cert\") pod \"console-9d65d6646-hc5j5\" (UID: \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\") " pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:01.032351 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:01.032331    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aa052cb1-c5e2-465a-89f2-a53c960e51fb-service-ca\") pod \"console-9d65d6646-hc5j5\" (UID: \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\") " pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:01.032449 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:01.032378    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aa052cb1-c5e2-465a-89f2-a53c960e51fb-console-oauth-config\") pod \"console-9d65d6646-hc5j5\" (UID: \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\") " pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:01.089789 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:01.089746    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:01.133041 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:01.132932    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nhcv\" (UniqueName: \"kubernetes.io/projected/aa052cb1-c5e2-465a-89f2-a53c960e51fb-kube-api-access-6nhcv\") pod \"console-9d65d6646-hc5j5\" (UID: \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\") " pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:01.133041 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:01.133034    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa052cb1-c5e2-465a-89f2-a53c960e51fb-trusted-ca-bundle\") pod \"console-9d65d6646-hc5j5\" (UID: \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\") " pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:01.133195 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:01.133062    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aa052cb1-c5e2-465a-89f2-a53c960e51fb-console-serving-cert\") pod \"console-9d65d6646-hc5j5\" (UID: \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\") " pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:01.133195 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:01.133084    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aa052cb1-c5e2-465a-89f2-a53c960e51fb-service-ca\") pod \"console-9d65d6646-hc5j5\" (UID: \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\") " pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:01.133195 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:01.133145    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aa052cb1-c5e2-465a-89f2-a53c960e51fb-console-oauth-config\") pod \"console-9d65d6646-hc5j5\" (UID: \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\") " pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:01.133344 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:01.133192    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aa052cb1-c5e2-465a-89f2-a53c960e51fb-oauth-serving-cert\") pod \"console-9d65d6646-hc5j5\" (UID: \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\") " pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:01.133344 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:01.133219    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aa052cb1-c5e2-465a-89f2-a53c960e51fb-console-config\") pod \"console-9d65d6646-hc5j5\" (UID: \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\") " pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:01.133868 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:01.133831    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aa052cb1-c5e2-465a-89f2-a53c960e51fb-service-ca\") pod \"console-9d65d6646-hc5j5\" (UID: \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\") " pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:01.133868 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:01.133855    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aa052cb1-c5e2-465a-89f2-a53c960e51fb-console-config\") pod \"console-9d65d6646-hc5j5\" (UID: \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\") " pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:01.134094 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:01.133884    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aa052cb1-c5e2-465a-89f2-a53c960e51fb-oauth-serving-cert\") pod \"console-9d65d6646-hc5j5\" (UID: \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\") " pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:01.134287 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:01.134264    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa052cb1-c5e2-465a-89f2-a53c960e51fb-trusted-ca-bundle\") pod \"console-9d65d6646-hc5j5\" (UID: \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\") " pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:01.135640 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:01.135615    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aa052cb1-c5e2-465a-89f2-a53c960e51fb-console-oauth-config\") pod \"console-9d65d6646-hc5j5\" (UID: \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\") " pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:01.135750 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:01.135731    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aa052cb1-c5e2-465a-89f2-a53c960e51fb-console-serving-cert\") pod \"console-9d65d6646-hc5j5\" (UID: \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\") " pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:01.140840 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:01.140818    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nhcv\" (UniqueName: \"kubernetes.io/projected/aa052cb1-c5e2-465a-89f2-a53c960e51fb-kube-api-access-6nhcv\") pod \"console-9d65d6646-hc5j5\" (UID: \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\") " pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:01.143114 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:01.143090    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:01.182250 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:01.182215    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:01.312891 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:01.312867    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9d65d6646-hc5j5"]
Jul 30 19:26:01.315379 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:26:01.315348    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa052cb1_c5e2_465a_89f2_a53c960e51fb.slice/crio-da4002960faafd67418555e5e3192d905843091fc78c1b4e761761248145c079 WatchSource:0}: Error finding container da4002960faafd67418555e5e3192d905843091fc78c1b4e761761248145c079: Status 404 returned error can't find the container with id da4002960faafd67418555e5e3192d905843091fc78c1b4e761761248145c079
Jul 30 19:26:02.009125 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:02.009088    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9d65d6646-hc5j5" event={"ID":"aa052cb1-c5e2-465a-89f2-a53c960e51fb","Type":"ContainerStarted","Data":"da4002960faafd67418555e5e3192d905843091fc78c1b4e761761248145c079"}
Jul 30 19:26:03.075511 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:03.075460    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:03.126443 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:03.126393    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:03.667209 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:03.667157    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:03.768370 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:03.768328    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:04.018774 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:04.018735    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9d65d6646-hc5j5" event={"ID":"aa052cb1-c5e2-465a-89f2-a53c960e51fb","Type":"ContainerStarted","Data":"865e8cc900bb8fe3b627774bb090afef3e051fb3985d89ff4c6e39892f21e1a2"}
Jul 30 19:26:04.038355 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:04.038294    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-9d65d6646-hc5j5" podStartSLOduration=1.439942733 podStartE2EDuration="4.038277666s" totalImagesPullingTime="2.598334933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:26:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:26:04.036925407 +0000 UTC m=+92.001506504" watchObservedRunningTime="2026-07-30 19:26:04.038277666 +0000 UTC m=+92.002858746"
Jul 30 19:26:05.629155 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:05.629120    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:07.084720 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.084684    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:07.251118 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.251089    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:07.346482 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.346398    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xlpnn"]
Jul 30 19:26:07.348986 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.348947    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.352662 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.352637    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 30 19:26:07.352791 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.352741    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 30 19:26:07.352791 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.352764    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 30 19:26:07.352892 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.352787    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 30 19:26:07.353882 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.353865    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-twtn8\""
Jul 30 19:26:07.375256 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.375200    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:07.438307 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.438273    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:07.486220 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.486179    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1de08b12-083f-413c-aca5-e4868ed0b718-sys\") pod \"node-exporter-xlpnn\" (UID: \"1de08b12-083f-413c-aca5-e4868ed0b718\") " pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.486220 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.486218    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1de08b12-083f-413c-aca5-e4868ed0b718-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xlpnn\" (UID: \"1de08b12-083f-413c-aca5-e4868ed0b718\") " pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.486462 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.486241    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1de08b12-083f-413c-aca5-e4868ed0b718-node-exporter-tls\") pod \"node-exporter-xlpnn\" (UID: \"1de08b12-083f-413c-aca5-e4868ed0b718\") " pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.486462 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.486284    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56mzm\" (UniqueName: \"kubernetes.io/projected/1de08b12-083f-413c-aca5-e4868ed0b718-kube-api-access-56mzm\") pod \"node-exporter-xlpnn\" (UID: \"1de08b12-083f-413c-aca5-e4868ed0b718\") " pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.486462 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.486306    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1de08b12-083f-413c-aca5-e4868ed0b718-node-exporter-textfile\") pod \"node-exporter-xlpnn\" (UID: \"1de08b12-083f-413c-aca5-e4868ed0b718\") " pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.486462 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.486356    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1de08b12-083f-413c-aca5-e4868ed0b718-metrics-client-ca\") pod \"node-exporter-xlpnn\" (UID: \"1de08b12-083f-413c-aca5-e4868ed0b718\") " pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.486462 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.486376    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1de08b12-083f-413c-aca5-e4868ed0b718-node-exporter-wtmp\") pod \"node-exporter-xlpnn\" (UID: \"1de08b12-083f-413c-aca5-e4868ed0b718\") " pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.486462 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.486393    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1de08b12-083f-413c-aca5-e4868ed0b718-node-exporter-accelerators-collector-config\") pod \"node-exporter-xlpnn\" (UID: \"1de08b12-083f-413c-aca5-e4868ed0b718\") " pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.486462 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.486434    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1de08b12-083f-413c-aca5-e4868ed0b718-root\") pod \"node-exporter-xlpnn\" (UID: \"1de08b12-083f-413c-aca5-e4868ed0b718\") " pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.538609 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.538580    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:07.591663 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.587623    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56mzm\" (UniqueName: \"kubernetes.io/projected/1de08b12-083f-413c-aca5-e4868ed0b718-kube-api-access-56mzm\") pod \"node-exporter-xlpnn\" (UID: \"1de08b12-083f-413c-aca5-e4868ed0b718\") " pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.591663 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.587684    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1de08b12-083f-413c-aca5-e4868ed0b718-node-exporter-textfile\") pod \"node-exporter-xlpnn\" (UID: \"1de08b12-083f-413c-aca5-e4868ed0b718\") " pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.591663 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.587725    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1de08b12-083f-413c-aca5-e4868ed0b718-metrics-client-ca\") pod \"node-exporter-xlpnn\" (UID: \"1de08b12-083f-413c-aca5-e4868ed0b718\") " pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.591663 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.587773    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1de08b12-083f-413c-aca5-e4868ed0b718-node-exporter-wtmp\") pod \"node-exporter-xlpnn\" (UID: \"1de08b12-083f-413c-aca5-e4868ed0b718\") " pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.591663 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.587804    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1de08b12-083f-413c-aca5-e4868ed0b718-node-exporter-accelerators-collector-config\") pod \"node-exporter-xlpnn\" (UID: \"1de08b12-083f-413c-aca5-e4868ed0b718\") " pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.591663 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.587857    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1de08b12-083f-413c-aca5-e4868ed0b718-root\") pod \"node-exporter-xlpnn\" (UID: \"1de08b12-083f-413c-aca5-e4868ed0b718\") " pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.591663 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.587937    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1de08b12-083f-413c-aca5-e4868ed0b718-sys\") pod \"node-exporter-xlpnn\" (UID: \"1de08b12-083f-413c-aca5-e4868ed0b718\") " pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.591663 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.587979    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1de08b12-083f-413c-aca5-e4868ed0b718-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xlpnn\" (UID: \"1de08b12-083f-413c-aca5-e4868ed0b718\") " pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.591663 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.588022    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1de08b12-083f-413c-aca5-e4868ed0b718-node-exporter-tls\") pod \"node-exporter-xlpnn\" (UID: \"1de08b12-083f-413c-aca5-e4868ed0b718\") " pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.591663 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.591059    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1de08b12-083f-413c-aca5-e4868ed0b718-node-exporter-tls\") pod \"node-exporter-xlpnn\" (UID: \"1de08b12-083f-413c-aca5-e4868ed0b718\") " pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.592216 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.591705    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1de08b12-083f-413c-aca5-e4868ed0b718-node-exporter-textfile\") pod \"node-exporter-xlpnn\" (UID: \"1de08b12-083f-413c-aca5-e4868ed0b718\") " pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.592216 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.591710    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1de08b12-083f-413c-aca5-e4868ed0b718-root\") pod \"node-exporter-xlpnn\" (UID: \"1de08b12-083f-413c-aca5-e4868ed0b718\") " pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.592216 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.591938    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1de08b12-083f-413c-aca5-e4868ed0b718-sys\") pod \"node-exporter-xlpnn\" (UID: \"1de08b12-083f-413c-aca5-e4868ed0b718\") " pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.592549 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.592510    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1de08b12-083f-413c-aca5-e4868ed0b718-node-exporter-wtmp\") pod \"node-exporter-xlpnn\" (UID: \"1de08b12-083f-413c-aca5-e4868ed0b718\") " pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.592772 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.592752    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1de08b12-083f-413c-aca5-e4868ed0b718-node-exporter-accelerators-collector-config\") pod \"node-exporter-xlpnn\" (UID: \"1de08b12-083f-413c-aca5-e4868ed0b718\") " pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.592851 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.592783    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1de08b12-083f-413c-aca5-e4868ed0b718-metrics-client-ca\") pod \"node-exporter-xlpnn\" (UID: \"1de08b12-083f-413c-aca5-e4868ed0b718\") " pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.594376 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.594351    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1de08b12-083f-413c-aca5-e4868ed0b718-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xlpnn\" (UID: \"1de08b12-083f-413c-aca5-e4868ed0b718\") " pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.600405 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.600352    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56mzm\" (UniqueName: \"kubernetes.io/projected/1de08b12-083f-413c-aca5-e4868ed0b718-kube-api-access-56mzm\") pod \"node-exporter-xlpnn\" (UID: \"1de08b12-083f-413c-aca5-e4868ed0b718\") " pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.638862 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.638823    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:07.657877 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.657838    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xlpnn"
Jul 30 19:26:07.669329 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:26:07.669301    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1de08b12_083f_413c_aca5_e4868ed0b718.slice/crio-89ba2cf9b66d3e824fb928c9024bbcc69c8185e5d6f3a95f4d7703afdb561ae5 WatchSource:0}: Error finding container 89ba2cf9b66d3e824fb928c9024bbcc69c8185e5d6f3a95f4d7703afdb561ae5: Status 404 returned error can't find the container with id 89ba2cf9b66d3e824fb928c9024bbcc69c8185e5d6f3a95f4d7703afdb561ae5
Jul 30 19:26:07.737303 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.737269    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:07.887536 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.887447    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:07.987847 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:07.987810    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:08.033716 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.033682    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xlpnn" event={"ID":"1de08b12-083f-413c-aca5-e4868ed0b718","Type":"ContainerStarted","Data":"89ba2cf9b66d3e824fb928c9024bbcc69c8185e5d6f3a95f4d7703afdb561ae5"}
Jul 30 19:26:08.312647 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.312614    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:08.399347 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.399310    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:08.404758 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.404729    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 19:26:08.408748 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.408725    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.411620 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.411549    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 30 19:26:08.411620 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.411602    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 30 19:26:08.411842 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.411602    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 30 19:26:08.411842 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.411651    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 30 19:26:08.411975 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.411887    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 30 19:26:08.412251 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.412233    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 30 19:26:08.412251 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.412245    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 30 19:26:08.412382 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.412278    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 30 19:26:08.412382 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.412248    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-xm54g\""
Jul 30 19:26:08.412552 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.412539    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 30 19:26:08.426141 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.426106    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 19:26:08.498311 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.498267    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cde68457-c34f-4caf-ad34-f70bcc61bc2b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.498487 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.498330    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.498487 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.498365    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnf7r\" (UniqueName: \"kubernetes.io/projected/cde68457-c34f-4caf-ad34-f70bcc61bc2b-kube-api-access-qnf7r\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.498487 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.498393    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.498487 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.498462    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cde68457-c34f-4caf-ad34-f70bcc61bc2b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.498487 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.498481    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cde68457-c34f-4caf-ad34-f70bcc61bc2b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.498743 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.498500    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.498743 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.498624    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.498743 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.498670    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cde68457-c34f-4caf-ad34-f70bcc61bc2b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.498899 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.498776    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.498899 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.498807    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cde68457-c34f-4caf-ad34-f70bcc61bc2b-config-out\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.498899 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.498858    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-config-volume\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.499089 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.498899    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-web-config\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.519143 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.519105    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:08.600113 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.600022    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.600113 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.600068    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cde68457-c34f-4caf-ad34-f70bcc61bc2b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.600113 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.600094    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.600419 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.600189    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cde68457-c34f-4caf-ad34-f70bcc61bc2b-config-out\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.600419 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.600252    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-config-volume\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.600419 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.600281    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-web-config\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.600419 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.600322    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cde68457-c34f-4caf-ad34-f70bcc61bc2b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.600419 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.600356    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.600419 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.600390    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnf7r\" (UniqueName: \"kubernetes.io/projected/cde68457-c34f-4caf-ad34-f70bcc61bc2b-kube-api-access-qnf7r\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.600758 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.600421    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.600758 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.600459    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cde68457-c34f-4caf-ad34-f70bcc61bc2b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.600758 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.600484    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cde68457-c34f-4caf-ad34-f70bcc61bc2b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.600758 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.600515    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.601047 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.601023    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cde68457-c34f-4caf-ad34-f70bcc61bc2b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.601550 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.601271    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cde68457-c34f-4caf-ad34-f70bcc61bc2b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.602504 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.602476    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cde68457-c34f-4caf-ad34-f70bcc61bc2b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.605069 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.605023    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cde68457-c34f-4caf-ad34-f70bcc61bc2b-config-out\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.605210 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.605175    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.605656 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.605607    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.605656 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.605623    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.605820 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.605759    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.605820 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.605781    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.605927 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.605879    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-web-config\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.607274 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.607251    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-config-volume\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.607602 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.607579    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cde68457-c34f-4caf-ad34-f70bcc61bc2b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.615813 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.615776    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnf7r\" (UniqueName: \"kubernetes.io/projected/cde68457-c34f-4caf-ad34-f70bcc61bc2b-kube-api-access-qnf7r\") pod \"alertmanager-main-0\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.695331 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.695171    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5bcbd966f6-xd72m"]
Jul 30 19:26:08.718686 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.718640    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:26:08.772919 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.772885    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:08.875853 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.875755    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:08.876203 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.876106    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 19:26:08.894750 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:08.894715    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8nv66"
Jul 30 19:26:08.965576 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:26:08.965543    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcde68457_c34f_4caf_ad34_f70bcc61bc2b.slice/crio-a87da962cee60f56feda392e031af26a21d7cf0ddc676d00e6e7d6e43c68084f WatchSource:0}: Error finding container a87da962cee60f56feda392e031af26a21d7cf0ddc676d00e6e7d6e43c68084f: Status 404 returned error can't find the container with id a87da962cee60f56feda392e031af26a21d7cf0ddc676d00e6e7d6e43c68084f
Jul 30 19:26:09.039690 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:09.039641    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cde68457-c34f-4caf-ad34-f70bcc61bc2b","Type":"ContainerStarted","Data":"a87da962cee60f56feda392e031af26a21d7cf0ddc676d00e6e7d6e43c68084f"}
Jul 30 19:26:09.323416 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:09.323377    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:09.375857 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:09.375822    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:09.784501 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:09.784467    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:09.885293 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:09.885256    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:10.046818 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:10.046740    2554 generic.go:356] "Generic (PLEG): container finished" podID="1de08b12-083f-413c-aca5-e4868ed0b718" containerID="f32f8a72f0934c2256153690a83707523aeef4a8d9ca1671f30ef9822a12503c" exitCode=0
Jul 30 19:26:10.046994 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:10.046829    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xlpnn" event={"ID":"1de08b12-083f-413c-aca5-e4868ed0b718","Type":"ContainerDied","Data":"f32f8a72f0934c2256153690a83707523aeef4a8d9ca1671f30ef9822a12503c"}
Jul 30 19:26:10.318325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:10.318293    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:10.416666 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:10.416629    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:10.472390 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:10.472359    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:10.526102 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:10.526067    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:11.052415 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.052379    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xlpnn" event={"ID":"1de08b12-083f-413c-aca5-e4868ed0b718","Type":"ContainerStarted","Data":"62bd2f6f5dc535925d808a07182cf46b744bea1ec84a283559da2eeada61b1db"}
Jul 30 19:26:11.052608 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.052449    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xlpnn" event={"ID":"1de08b12-083f-413c-aca5-e4868ed0b718","Type":"ContainerStarted","Data":"13f6c706cd8925fff3b80f35f892f59e55660d07581a395782bd1525c1a0b3bf"}
Jul 30 19:26:11.053882 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.053852    2554 generic.go:356] "Generic (PLEG): container finished" podID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerID="7398ad818904879a9bf2ec6f959cea4cebdb68af664cc686a7ac60517dfb21b5" exitCode=0
Jul 30 19:26:11.053991 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.053913    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cde68457-c34f-4caf-ad34-f70bcc61bc2b","Type":"ContainerDied","Data":"7398ad818904879a9bf2ec6f959cea4cebdb68af664cc686a7ac60517dfb21b5"}
Jul 30 19:26:11.072584 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.072530    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xlpnn" podStartSLOduration=2.680965762 podStartE2EDuration="4.072514732s" totalImagesPullingTime="1.323065681s" totalInitContainerRuntime="68.483289ms" isStatefulPod=true podCreationTimestamp="2026-07-30 19:26:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:26:11.07023613 +0000 UTC m=+99.034817225" watchObservedRunningTime="2026-07-30 19:26:11.072514732 +0000 UTC m=+99.037095813"
Jul 30 19:26:11.182800 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.182748    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:11.183024 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.182827    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:11.187919 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.187894    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:11.387694 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.387566    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:11.487948 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.487903    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:11.651939 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.651858    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-59764589d-r26vv"]
Jul 30 19:26:11.653948 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.653929    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-59764589d-r26vv"
Jul 30 19:26:11.657574 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.657405    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 30 19:26:11.657574 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.657405    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 30 19:26:11.657574 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.657462    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-10b7fbumrq6up\""
Jul 30 19:26:11.657574 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.657469    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 30 19:26:11.657574 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.657543    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 30 19:26:11.657905 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.657574    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-qlkxc\""
Jul 30 19:26:11.664727 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.664705    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-59764589d-r26vv"]
Jul 30 19:26:11.728639 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.728601    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzqnb\" (UniqueName: \"kubernetes.io/projected/66e316b4-ad9c-43e6-a692-d675438f8af4-kube-api-access-fzqnb\") pod \"metrics-server-59764589d-r26vv\" (UID: \"66e316b4-ad9c-43e6-a692-d675438f8af4\") " pod="openshift-monitoring/metrics-server-59764589d-r26vv"
Jul 30 19:26:11.728815 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.728646    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66e316b4-ad9c-43e6-a692-d675438f8af4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-59764589d-r26vv\" (UID: \"66e316b4-ad9c-43e6-a692-d675438f8af4\") " pod="openshift-monitoring/metrics-server-59764589d-r26vv"
Jul 30 19:26:11.728815 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.728708    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/66e316b4-ad9c-43e6-a692-d675438f8af4-metrics-server-audit-profiles\") pod \"metrics-server-59764589d-r26vv\" (UID: \"66e316b4-ad9c-43e6-a692-d675438f8af4\") " pod="openshift-monitoring/metrics-server-59764589d-r26vv"
Jul 30 19:26:11.728815 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.728736    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/66e316b4-ad9c-43e6-a692-d675438f8af4-client-ca-bundle\") pod \"metrics-server-59764589d-r26vv\" (UID: \"66e316b4-ad9c-43e6-a692-d675438f8af4\") " pod="openshift-monitoring/metrics-server-59764589d-r26vv"
Jul 30 19:26:11.728815 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.728765    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/66e316b4-ad9c-43e6-a692-d675438f8af4-secret-metrics-server-client-certs\") pod \"metrics-server-59764589d-r26vv\" (UID: \"66e316b4-ad9c-43e6-a692-d675438f8af4\") " pod="openshift-monitoring/metrics-server-59764589d-r26vv"
Jul 30 19:26:11.728815 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.728797    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/66e316b4-ad9c-43e6-a692-d675438f8af4-secret-metrics-server-tls\") pod \"metrics-server-59764589d-r26vv\" (UID: \"66e316b4-ad9c-43e6-a692-d675438f8af4\") " pod="openshift-monitoring/metrics-server-59764589d-r26vv"
Jul 30 19:26:11.729035 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.728869    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/66e316b4-ad9c-43e6-a692-d675438f8af4-audit-log\") pod \"metrics-server-59764589d-r26vv\" (UID: \"66e316b4-ad9c-43e6-a692-d675438f8af4\") " pod="openshift-monitoring/metrics-server-59764589d-r26vv"
Jul 30 19:26:11.830333 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.830294    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/66e316b4-ad9c-43e6-a692-d675438f8af4-metrics-server-audit-profiles\") pod \"metrics-server-59764589d-r26vv\" (UID: \"66e316b4-ad9c-43e6-a692-d675438f8af4\") " pod="openshift-monitoring/metrics-server-59764589d-r26vv"
Jul 30 19:26:11.830531 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.830354    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/66e316b4-ad9c-43e6-a692-d675438f8af4-client-ca-bundle\") pod \"metrics-server-59764589d-r26vv\" (UID: \"66e316b4-ad9c-43e6-a692-d675438f8af4\") " pod="openshift-monitoring/metrics-server-59764589d-r26vv"
Jul 30 19:26:11.830531 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.830382    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/66e316b4-ad9c-43e6-a692-d675438f8af4-secret-metrics-server-client-certs\") pod \"metrics-server-59764589d-r26vv\" (UID: \"66e316b4-ad9c-43e6-a692-d675438f8af4\") " pod="openshift-monitoring/metrics-server-59764589d-r26vv"
Jul 30 19:26:11.830531 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.830424    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/66e316b4-ad9c-43e6-a692-d675438f8af4-secret-metrics-server-tls\") pod \"metrics-server-59764589d-r26vv\" (UID: \"66e316b4-ad9c-43e6-a692-d675438f8af4\") " pod="openshift-monitoring/metrics-server-59764589d-r26vv"
Jul 30 19:26:11.830531 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.830467    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/66e316b4-ad9c-43e6-a692-d675438f8af4-audit-log\") pod \"metrics-server-59764589d-r26vv\" (UID: \"66e316b4-ad9c-43e6-a692-d675438f8af4\") " pod="openshift-monitoring/metrics-server-59764589d-r26vv"
Jul 30 19:26:11.830746 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.830537    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzqnb\" (UniqueName: \"kubernetes.io/projected/66e316b4-ad9c-43e6-a692-d675438f8af4-kube-api-access-fzqnb\") pod \"metrics-server-59764589d-r26vv\" (UID: \"66e316b4-ad9c-43e6-a692-d675438f8af4\") " pod="openshift-monitoring/metrics-server-59764589d-r26vv"
Jul 30 19:26:11.830746 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.830573    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66e316b4-ad9c-43e6-a692-d675438f8af4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-59764589d-r26vv\" (UID: \"66e316b4-ad9c-43e6-a692-d675438f8af4\") " pod="openshift-monitoring/metrics-server-59764589d-r26vv"
Jul 30 19:26:11.831272 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.831219    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/66e316b4-ad9c-43e6-a692-d675438f8af4-audit-log\") pod \"metrics-server-59764589d-r26vv\" (UID: \"66e316b4-ad9c-43e6-a692-d675438f8af4\") " pod="openshift-monitoring/metrics-server-59764589d-r26vv"
Jul 30 19:26:11.831272 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.831249    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66e316b4-ad9c-43e6-a692-d675438f8af4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-59764589d-r26vv\" (UID: \"66e316b4-ad9c-43e6-a692-d675438f8af4\") " pod="openshift-monitoring/metrics-server-59764589d-r26vv"
Jul 30 19:26:11.831457 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.831393    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/66e316b4-ad9c-43e6-a692-d675438f8af4-metrics-server-audit-profiles\") pod \"metrics-server-59764589d-r26vv\" (UID: \"66e316b4-ad9c-43e6-a692-d675438f8af4\") " pod="openshift-monitoring/metrics-server-59764589d-r26vv"
Jul 30 19:26:11.833336 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.833312    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/66e316b4-ad9c-43e6-a692-d675438f8af4-client-ca-bundle\") pod \"metrics-server-59764589d-r26vv\" (UID: \"66e316b4-ad9c-43e6-a692-d675438f8af4\") " pod="openshift-monitoring/metrics-server-59764589d-r26vv"
Jul 30 19:26:11.833561 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.833537    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/66e316b4-ad9c-43e6-a692-d675438f8af4-secret-metrics-server-tls\") pod \"metrics-server-59764589d-r26vv\" (UID: \"66e316b4-ad9c-43e6-a692-d675438f8af4\") " pod="openshift-monitoring/metrics-server-59764589d-r26vv"
Jul 30 19:26:11.833782 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.833761    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/66e316b4-ad9c-43e6-a692-d675438f8af4-secret-metrics-server-client-certs\") pod \"metrics-server-59764589d-r26vv\" (UID: \"66e316b4-ad9c-43e6-a692-d675438f8af4\") " pod="openshift-monitoring/metrics-server-59764589d-r26vv"
Jul 30 19:26:11.842068 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.842045    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzqnb\" (UniqueName: \"kubernetes.io/projected/66e316b4-ad9c-43e6-a692-d675438f8af4-kube-api-access-fzqnb\") pod \"metrics-server-59764589d-r26vv\" (UID: \"66e316b4-ad9c-43e6-a692-d675438f8af4\") " pod="openshift-monitoring/metrics-server-59764589d-r26vv"
Jul 30 19:26:11.963913 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:11.963869    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-59764589d-r26vv"
Jul 30 19:26:12.066726 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:12.066611    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:12.111165 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:12.111129    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:12.111920 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:12.111900    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-59764589d-r26vv"]
Jul 30 19:26:12.114029 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:26:12.114001    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66e316b4_ad9c_43e6_a692_d675438f8af4.slice/crio-bc29519a14630ab341605b727ead382e1ea7c9fb5aac79c678f3263202a60ccc WatchSource:0}: Error finding container bc29519a14630ab341605b727ead382e1ea7c9fb5aac79c678f3263202a60ccc: Status 404 returned error can't find the container with id bc29519a14630ab341605b727ead382e1ea7c9fb5aac79c678f3263202a60ccc
Jul 30 19:26:12.168055 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:12.168017    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:12.324353 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:12.324239    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:12.800267 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:12.800216    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:12.899165 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:12.899132    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:13.067196 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:13.067155    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cde68457-c34f-4caf-ad34-f70bcc61bc2b","Type":"ContainerStarted","Data":"b1aff326ca513262e2baf9f1056d7d73c2280ada8a16e3c87a4b7731fabeb5a2"}
Jul 30 19:26:13.067349 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:13.067235    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cde68457-c34f-4caf-ad34-f70bcc61bc2b","Type":"ContainerStarted","Data":"f412e1c4a70c8c483c5761b331e7cd6e9c6ea41c66a5d3a65aed5b12c2457af7"}
Jul 30 19:26:13.068736 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:13.068711    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-59764589d-r26vv" event={"ID":"66e316b4-ad9c-43e6-a692-d675438f8af4","Type":"ContainerStarted","Data":"bc29519a14630ab341605b727ead382e1ea7c9fb5aac79c678f3263202a60ccc"}
Jul 30 19:26:13.081372 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:13.081342    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:13.181907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:13.181857    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:13.684087 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:13.684041    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:14.076609 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:14.076576    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cde68457-c34f-4caf-ad34-f70bcc61bc2b","Type":"ContainerStarted","Data":"0c311d55e20bc96d15126a632cf106cfb17fa1426cf2043b7512fc5b1d5cfd85"}
Jul 30 19:26:14.077006 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:14.076644    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cde68457-c34f-4caf-ad34-f70bcc61bc2b","Type":"ContainerStarted","Data":"a980a8c3908293089d8c9815722a2daff74ced959e3a29a0236c450bff24ca8f"}
Jul 30 19:26:14.077006 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:14.076671    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cde68457-c34f-4caf-ad34-f70bcc61bc2b","Type":"ContainerStarted","Data":"c998a524cafe1ccd424e7162b0d8f7a64161975bb4a33bd94fbeae506f461975"}
Jul 30 19:26:14.138137 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:14.138104    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:14.239532 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:14.239502    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:15.084627 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:15.084591    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cde68457-c34f-4caf-ad34-f70bcc61bc2b","Type":"ContainerStarted","Data":"d786666e53ed2cf1642c41e661dfc6fafafb3d5f5506304c1f230a49d5bc41e3"}
Jul 30 19:26:15.086196 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:15.086169    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-59764589d-r26vv" event={"ID":"66e316b4-ad9c-43e6-a692-d675438f8af4","Type":"ContainerStarted","Data":"74618c9c92027641e672bd8fe3b5c801b98f05caff323cd9189a3f520fcade06"}
Jul 30 19:26:15.119621 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:15.119559    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.96568047 podStartE2EDuration="7.11954242s" totalImagesPullingTime="4.133334172s" totalInitContainerRuntime="20.527778ms" isStatefulPod=false podCreationTimestamp="2026-07-30 19:26:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:26:15.119127765 +0000 UTC m=+103.083708851" watchObservedRunningTime="2026-07-30 19:26:15.11954242 +0000 UTC m=+103.084123500"
Jul 30 19:26:15.140819 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:15.140289    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-59764589d-r26vv" podStartSLOduration=2.174241677 podStartE2EDuration="4.140267575s" totalImagesPullingTime="1.966025898s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:26:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:26:15.139838988 +0000 UTC m=+103.104420070" watchObservedRunningTime="2026-07-30 19:26:15.140267575 +0000 UTC m=+103.104848657"
Jul 30 19:26:18.886275 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:18.886239    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:18.965420 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:18.965384    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:21.893416 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:21.893381    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:25.126905 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:25.126862    2554 generic.go:356] "Generic (PLEG): container finished" podID="083469f7-657e-4dc8-98ac-70c9749d5922" containerID="86665d66c58f0994ad816babdae3a412b80241c78b2e6bc74b3802e6028f28c5" exitCode=0
Jul 30 19:26:25.127401 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:25.126914    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-72ngp" event={"ID":"083469f7-657e-4dc8-98ac-70c9749d5922","Type":"ContainerDied","Data":"86665d66c58f0994ad816babdae3a412b80241c78b2e6bc74b3802e6028f28c5"}
Jul 30 19:26:25.127401 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:25.127361    2554 scope.go:117] "RemoveContainer" containerID="86665d66c58f0994ad816babdae3a412b80241c78b2e6bc74b3802e6028f28c5"
Jul 30 19:26:26.133229 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:26.133197    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-72ngp" event={"ID":"083469f7-657e-4dc8-98ac-70c9749d5922","Type":"ContainerStarted","Data":"fdb80befb1fe55fe34e76daba7c26020cc8b6846d34ac24e44a16c4826d14dbd"}
Jul 30 19:26:26.377340 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:26.377307    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:26.427120 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:26.427033    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:29.930066 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:29.930023    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9d65d6646-hc5j5"]
Jul 30 19:26:30.026368 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:30.026329    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:30.150920 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:30.150885    2554 generic.go:356] "Generic (PLEG): container finished" podID="a9c42650-7ee2-42f7-9b2d-dbf601ba95b3" containerID="905647d8ac9e0bd4908189caab448beecd0976c9b9c669813e972cba90ba02b5" exitCode=0
Jul 30 19:26:30.151108 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:30.150981    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-msg4s" event={"ID":"a9c42650-7ee2-42f7-9b2d-dbf601ba95b3","Type":"ContainerDied","Data":"905647d8ac9e0bd4908189caab448beecd0976c9b9c669813e972cba90ba02b5"}
Jul 30 19:26:30.151347 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:30.151332    2554 scope.go:117] "RemoveContainer" containerID="905647d8ac9e0bd4908189caab448beecd0976c9b9c669813e972cba90ba02b5"
Jul 30 19:26:31.156259 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:31.156227    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-msg4s" event={"ID":"a9c42650-7ee2-42f7-9b2d-dbf601ba95b3","Type":"ContainerStarted","Data":"a2f9d4617cfc604ab67d958eaa4066f16e5e75c3803895474d35a64dce655e7b"}
Jul 30 19:26:31.964227 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:31.964187    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-59764589d-r26vv"
Jul 30 19:26:31.964404 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:31.964280    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-59764589d-r26vv"
Jul 30 19:26:33.719916 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:33.719872    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m" podUID="bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf" containerName="registry" containerID="cri-o://b82e67e08f73a258195be6620bd617068cd4f51f911a00074ed4824735141ee2" gracePeriod=30
Jul 30 19:26:33.962580 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:33.962550    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:26:34.032242 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.032151    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-bound-sa-token\") pod \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") "
Jul 30 19:26:34.032242 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.032208    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-tls\") pod \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") "
Jul 30 19:26:34.032457 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.032243    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-image-registry-private-configuration\") pod \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") "
Jul 30 19:26:34.032457 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.032341    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-certificates\") pod \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") "
Jul 30 19:26:34.032457 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.032384    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ljcfq\" (UniqueName: \"kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-kube-api-access-ljcfq\") pod \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") "
Jul 30 19:26:34.032457 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.032411    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-ca-trust-extracted\") pod \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") "
Jul 30 19:26:34.032653 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.032562    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-trusted-ca\") pod \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") "
Jul 30 19:26:34.032715 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.032661    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-installation-pull-secrets\") pod \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\" (UID: \"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf\") "
Jul 30 19:26:34.032802 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.032775    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf" (UID: "bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 19:26:34.033010 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.032970    2554 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-certificates\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:26:34.033193 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.033163    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf" (UID: "bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 19:26:34.034883 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.034851    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf" (UID: "bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:26:34.035054 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.035000    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf" (UID: "bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 19:26:34.035107 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.035074    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf" (UID: "bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:26:34.035107 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.035086    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf" (UID: "bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 19:26:34.035336 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.035315    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-kube-api-access-ljcfq" (OuterVolumeSpecName: "kube-api-access-ljcfq") pod "bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf" (UID: "bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf"). InnerVolumeSpecName "kube-api-access-ljcfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:26:34.043371 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.043339    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf" (UID: "bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:26:34.133870 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.133825    2554 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-image-registry-private-configuration\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:26:34.133870 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.133861    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljcfq\" (UniqueName: \"kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-kube-api-access-ljcfq\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:26:34.133870 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.133871    2554 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-ca-trust-extracted\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:26:34.133870 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.133881    2554 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-trusted-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:26:34.134163 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.133890    2554 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-installation-pull-secrets\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:26:34.134163 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.133900    2554 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-bound-sa-token\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:26:34.134163 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.133908    2554 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf-registry-tls\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:26:34.168379 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.168344    2554 generic.go:356] "Generic (PLEG): container finished" podID="bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf" containerID="b82e67e08f73a258195be6620bd617068cd4f51f911a00074ed4824735141ee2" exitCode=0
Jul 30 19:26:34.168523 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.168390    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m"
Jul 30 19:26:34.168523 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.168436    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m" event={"ID":"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf","Type":"ContainerDied","Data":"b82e67e08f73a258195be6620bd617068cd4f51f911a00074ed4824735141ee2"}
Jul 30 19:26:34.168523 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.168488    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bcbd966f6-xd72m" event={"ID":"bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf","Type":"ContainerDied","Data":"d73ffc9e82a6f42671522b1e40a6515fe2b22220ea219c0196531c81e91fa6ab"}
Jul 30 19:26:34.168523 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.168515    2554 scope.go:117] "RemoveContainer" containerID="b82e67e08f73a258195be6620bd617068cd4f51f911a00074ed4824735141ee2"
Jul 30 19:26:34.177672 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.177651    2554 scope.go:117] "RemoveContainer" containerID="b82e67e08f73a258195be6620bd617068cd4f51f911a00074ed4824735141ee2"
Jul 30 19:26:34.177993 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:26:34.177939    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b82e67e08f73a258195be6620bd617068cd4f51f911a00074ed4824735141ee2\": container with ID starting with b82e67e08f73a258195be6620bd617068cd4f51f911a00074ed4824735141ee2 not found: ID does not exist" containerID="b82e67e08f73a258195be6620bd617068cd4f51f911a00074ed4824735141ee2"
Jul 30 19:26:34.178045 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.178000    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b82e67e08f73a258195be6620bd617068cd4f51f911a00074ed4824735141ee2"} err="failed to get container status \"b82e67e08f73a258195be6620bd617068cd4f51f911a00074ed4824735141ee2\": rpc error: code = NotFound desc = could not find container \"b82e67e08f73a258195be6620bd617068cd4f51f911a00074ed4824735141ee2\": container with ID starting with b82e67e08f73a258195be6620bd617068cd4f51f911a00074ed4824735141ee2 not found: ID does not exist"
Jul 30 19:26:34.189051 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.189025    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5bcbd966f6-xd72m"]
Jul 30 19:26:34.192898 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.192876    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5bcbd966f6-xd72m"]
Jul 30 19:26:34.489239 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:34.486429    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf" path="/var/lib/kubelet/pods/bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf/volumes"
Jul 30 19:26:35.706126 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:35.706092    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:41.386016 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:41.385976    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:41.438528 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:41.438493    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:44.785622 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:44.785585    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:44.892636 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:44.892603    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:44.949702 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:44.949627    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:26:45.212699 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:45.212671    2554 generic.go:356] "Generic (PLEG): container finished" podID="471ab30c-b470-404b-b10a-d72642865017" containerID="4d68c018c3a86f3cbbf710acfd42d06d52bf11d0888159952f2f93d805ed5612" exitCode=0
Jul 30 19:26:45.212853 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:45.212705    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-smtsw" event={"ID":"471ab30c-b470-404b-b10a-d72642865017","Type":"ContainerDied","Data":"4d68c018c3a86f3cbbf710acfd42d06d52bf11d0888159952f2f93d805ed5612"}
Jul 30 19:26:45.213040 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:45.213024    2554 scope.go:117] "RemoveContainer" containerID="4d68c018c3a86f3cbbf710acfd42d06d52bf11d0888159952f2f93d805ed5612"
Jul 30 19:26:46.218762 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:46.218726    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-smtsw" event={"ID":"471ab30c-b470-404b-b10a-d72642865017","Type":"ContainerStarted","Data":"e78acbd9b946b50668bf40bfa105b57eec986945a4bbedca5b93580e67eda387"}
Jul 30 19:26:51.969969 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:51.969932    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-59764589d-r26vv"
Jul 30 19:26:51.974263 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:51.974232    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-59764589d-r26vv"
Jul 30 19:26:54.951572 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:54.951527    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-9d65d6646-hc5j5" podUID="aa052cb1-c5e2-465a-89f2-a53c960e51fb" containerName="console" containerID="cri-o://865e8cc900bb8fe3b627774bb090afef3e051fb3985d89ff4c6e39892f21e1a2" gracePeriod=15
Jul 30 19:26:55.200304 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.200276    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:55.256472 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.256440    2554 generic.go:356] "Generic (PLEG): container finished" podID="aa052cb1-c5e2-465a-89f2-a53c960e51fb" containerID="865e8cc900bb8fe3b627774bb090afef3e051fb3985d89ff4c6e39892f21e1a2" exitCode=0
Jul 30 19:26:55.256669 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.256493    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9d65d6646-hc5j5"
Jul 30 19:26:55.256669 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.256526    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9d65d6646-hc5j5" event={"ID":"aa052cb1-c5e2-465a-89f2-a53c960e51fb","Type":"ContainerDied","Data":"865e8cc900bb8fe3b627774bb090afef3e051fb3985d89ff4c6e39892f21e1a2"}
Jul 30 19:26:55.256669 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.256577    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9d65d6646-hc5j5" event={"ID":"aa052cb1-c5e2-465a-89f2-a53c960e51fb","Type":"ContainerDied","Data":"da4002960faafd67418555e5e3192d905843091fc78c1b4e761761248145c079"}
Jul 30 19:26:55.256669 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.256600    2554 scope.go:117] "RemoveContainer" containerID="865e8cc900bb8fe3b627774bb090afef3e051fb3985d89ff4c6e39892f21e1a2"
Jul 30 19:26:55.264739 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.264719    2554 scope.go:117] "RemoveContainer" containerID="865e8cc900bb8fe3b627774bb090afef3e051fb3985d89ff4c6e39892f21e1a2"
Jul 30 19:26:55.265058 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:26:55.265037    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"865e8cc900bb8fe3b627774bb090afef3e051fb3985d89ff4c6e39892f21e1a2\": container with ID starting with 865e8cc900bb8fe3b627774bb090afef3e051fb3985d89ff4c6e39892f21e1a2 not found: ID does not exist" containerID="865e8cc900bb8fe3b627774bb090afef3e051fb3985d89ff4c6e39892f21e1a2"
Jul 30 19:26:55.265114 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.265068    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"865e8cc900bb8fe3b627774bb090afef3e051fb3985d89ff4c6e39892f21e1a2"} err="failed to get container status \"865e8cc900bb8fe3b627774bb090afef3e051fb3985d89ff4c6e39892f21e1a2\": rpc error: code = NotFound desc = could not find container \"865e8cc900bb8fe3b627774bb090afef3e051fb3985d89ff4c6e39892f21e1a2\": container with ID starting with 865e8cc900bb8fe3b627774bb090afef3e051fb3985d89ff4c6e39892f21e1a2 not found: ID does not exist"
Jul 30 19:26:55.317566 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.317523    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aa052cb1-c5e2-465a-89f2-a53c960e51fb-console-serving-cert\") pod \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\" (UID: \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\") "
Jul 30 19:26:55.317752 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.317589    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nhcv\" (UniqueName: \"kubernetes.io/projected/aa052cb1-c5e2-465a-89f2-a53c960e51fb-kube-api-access-6nhcv\") pod \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\" (UID: \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\") "
Jul 30 19:26:55.317752 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.317625    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aa052cb1-c5e2-465a-89f2-a53c960e51fb-console-config\") pod \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\" (UID: \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\") "
Jul 30 19:26:55.317752 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.317680    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aa052cb1-c5e2-465a-89f2-a53c960e51fb-oauth-serving-cert\") pod \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\" (UID: \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\") "
Jul 30 19:26:55.317752 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.317706    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aa052cb1-c5e2-465a-89f2-a53c960e51fb-service-ca\") pod \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\" (UID: \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\") "
Jul 30 19:26:55.317752 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.317726    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa052cb1-c5e2-465a-89f2-a53c960e51fb-trusted-ca-bundle\") pod \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\" (UID: \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\") "
Jul 30 19:26:55.318027 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.317776    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aa052cb1-c5e2-465a-89f2-a53c960e51fb-console-oauth-config\") pod \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\" (UID: \"aa052cb1-c5e2-465a-89f2-a53c960e51fb\") "
Jul 30 19:26:55.318172 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.318104    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa052cb1-c5e2-465a-89f2-a53c960e51fb-console-config" (OuterVolumeSpecName: "console-config") pod "aa052cb1-c5e2-465a-89f2-a53c960e51fb" (UID: "aa052cb1-c5e2-465a-89f2-a53c960e51fb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 19:26:55.318298 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.318210    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa052cb1-c5e2-465a-89f2-a53c960e51fb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "aa052cb1-c5e2-465a-89f2-a53c960e51fb" (UID: "aa052cb1-c5e2-465a-89f2-a53c960e51fb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 19:26:55.318298 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.318230    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa052cb1-c5e2-465a-89f2-a53c960e51fb-service-ca" (OuterVolumeSpecName: "service-ca") pod "aa052cb1-c5e2-465a-89f2-a53c960e51fb" (UID: "aa052cb1-c5e2-465a-89f2-a53c960e51fb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 19:26:55.318405 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.318379    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa052cb1-c5e2-465a-89f2-a53c960e51fb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "aa052cb1-c5e2-465a-89f2-a53c960e51fb" (UID: "aa052cb1-c5e2-465a-89f2-a53c960e51fb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 19:26:55.319906 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.319882    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa052cb1-c5e2-465a-89f2-a53c960e51fb-kube-api-access-6nhcv" (OuterVolumeSpecName: "kube-api-access-6nhcv") pod "aa052cb1-c5e2-465a-89f2-a53c960e51fb" (UID: "aa052cb1-c5e2-465a-89f2-a53c960e51fb"). InnerVolumeSpecName "kube-api-access-6nhcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:26:55.320020 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.319905    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa052cb1-c5e2-465a-89f2-a53c960e51fb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "aa052cb1-c5e2-465a-89f2-a53c960e51fb" (UID: "aa052cb1-c5e2-465a-89f2-a53c960e51fb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 19:26:55.320020 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.319938    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa052cb1-c5e2-465a-89f2-a53c960e51fb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "aa052cb1-c5e2-465a-89f2-a53c960e51fb" (UID: "aa052cb1-c5e2-465a-89f2-a53c960e51fb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 19:26:55.418397 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.418353    2554 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aa052cb1-c5e2-465a-89f2-a53c960e51fb-oauth-serving-cert\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:26:55.418397 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.418392    2554 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aa052cb1-c5e2-465a-89f2-a53c960e51fb-service-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:26:55.418397 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.418402    2554 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa052cb1-c5e2-465a-89f2-a53c960e51fb-trusted-ca-bundle\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:26:55.418397 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.418410    2554 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aa052cb1-c5e2-465a-89f2-a53c960e51fb-console-oauth-config\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:26:55.418665 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.418419    2554 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aa052cb1-c5e2-465a-89f2-a53c960e51fb-console-serving-cert\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:26:55.418665 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.418427    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nhcv\" (UniqueName: \"kubernetes.io/projected/aa052cb1-c5e2-465a-89f2-a53c960e51fb-kube-api-access-6nhcv\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:26:55.418665 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.418436    2554 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aa052cb1-c5e2-465a-89f2-a53c960e51fb-console-config\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:26:55.577683 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.577650    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9d65d6646-hc5j5"]
Jul 30 19:26:55.579844 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:55.579816    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-9d65d6646-hc5j5"]
Jul 30 19:26:56.487635 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:26:56.487597    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa052cb1-c5e2-465a-89f2-a53c960e51fb" path="/var/lib/kubelet/pods/aa052cb1-c5e2-465a-89f2-a53c960e51fb/volumes"
Jul 30 19:27:15.076141 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:15.076104    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:27:16.431598 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:16.431563    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:27:23.472446 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:23.472404    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:27:23.527623 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:23.527587    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:27:27.747216 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:27.747119    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 19:27:27.747710 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:27.747522    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerName="alertmanager" containerID="cri-o://f412e1c4a70c8c483c5761b331e7cd6e9c6ea41c66a5d3a65aed5b12c2457af7" gracePeriod=120
Jul 30 19:27:27.747710 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:27.747573    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerName="kube-rbac-proxy-metric" containerID="cri-o://0c311d55e20bc96d15126a632cf106cfb17fa1426cf2043b7512fc5b1d5cfd85" gracePeriod=120
Jul 30 19:27:27.747710 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:27.747621    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerName="kube-rbac-proxy" containerID="cri-o://a980a8c3908293089d8c9815722a2daff74ced959e3a29a0236c450bff24ca8f" gracePeriod=120
Jul 30 19:27:27.747710 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:27.747641    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerName="config-reloader" containerID="cri-o://b1aff326ca513262e2baf9f1056d7d73c2280ada8a16e3c87a4b7731fabeb5a2" gracePeriod=120
Jul 30 19:27:27.747974 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:27.747603    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerName="kube-rbac-proxy-web" containerID="cri-o://c998a524cafe1ccd424e7162b0d8f7a64161975bb4a33bd94fbeae506f461975" gracePeriod=120
Jul 30 19:27:27.747974 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:27.747639    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerName="prom-label-proxy" containerID="cri-o://d786666e53ed2cf1642c41e661dfc6fafafb3d5f5506304c1f230a49d5bc41e3" gracePeriod=120
Jul 30 19:27:28.391432 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:28.391400    2554 generic.go:356] "Generic (PLEG): container finished" podID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerID="d786666e53ed2cf1642c41e661dfc6fafafb3d5f5506304c1f230a49d5bc41e3" exitCode=0
Jul 30 19:27:28.391432 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:28.391427    2554 generic.go:356] "Generic (PLEG): container finished" podID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerID="a980a8c3908293089d8c9815722a2daff74ced959e3a29a0236c450bff24ca8f" exitCode=0
Jul 30 19:27:28.391432 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:28.391434    2554 generic.go:356] "Generic (PLEG): container finished" podID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerID="b1aff326ca513262e2baf9f1056d7d73c2280ada8a16e3c87a4b7731fabeb5a2" exitCode=0
Jul 30 19:27:28.391432 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:28.391440    2554 generic.go:356] "Generic (PLEG): container finished" podID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerID="f412e1c4a70c8c483c5761b331e7cd6e9c6ea41c66a5d3a65aed5b12c2457af7" exitCode=0
Jul 30 19:27:28.391700 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:28.391466    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cde68457-c34f-4caf-ad34-f70bcc61bc2b","Type":"ContainerDied","Data":"d786666e53ed2cf1642c41e661dfc6fafafb3d5f5506304c1f230a49d5bc41e3"}
Jul 30 19:27:28.391700 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:28.391528    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cde68457-c34f-4caf-ad34-f70bcc61bc2b","Type":"ContainerDied","Data":"a980a8c3908293089d8c9815722a2daff74ced959e3a29a0236c450bff24ca8f"}
Jul 30 19:27:28.391700 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:28.391558    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cde68457-c34f-4caf-ad34-f70bcc61bc2b","Type":"ContainerDied","Data":"b1aff326ca513262e2baf9f1056d7d73c2280ada8a16e3c87a4b7731fabeb5a2"}
Jul 30 19:27:28.391700 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:28.391582    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cde68457-c34f-4caf-ad34-f70bcc61bc2b","Type":"ContainerDied","Data":"f412e1c4a70c8c483c5761b331e7cd6e9c6ea41c66a5d3a65aed5b12c2457af7"}
Jul 30 19:27:28.993477 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:28.993451    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.082641 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.082562    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cde68457-c34f-4caf-ad34-f70bcc61bc2b-tls-assets\") pod \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") "
Jul 30 19:27:29.082641 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.082600    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cde68457-c34f-4caf-ad34-f70bcc61bc2b-alertmanager-trusted-ca-bundle\") pod \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") "
Jul 30 19:27:29.082641 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.082627    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-secret-alertmanager-kube-rbac-proxy-web\") pod \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") "
Jul 30 19:27:29.082900 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.082812    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnf7r\" (UniqueName: \"kubernetes.io/projected/cde68457-c34f-4caf-ad34-f70bcc61bc2b-kube-api-access-qnf7r\") pod \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") "
Jul 30 19:27:29.082900 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.082880    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cde68457-c34f-4caf-ad34-f70bcc61bc2b-metrics-client-ca\") pod \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") "
Jul 30 19:27:29.083040 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.082924    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-secret-alertmanager-main-tls\") pod \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") "
Jul 30 19:27:29.083040 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.082965    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-cluster-tls-config\") pod \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") "
Jul 30 19:27:29.083040 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.082999    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-web-config\") pod \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") "
Jul 30 19:27:29.083040 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.083008    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cde68457-c34f-4caf-ad34-f70bcc61bc2b-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "cde68457-c34f-4caf-ad34-f70bcc61bc2b" (UID: "cde68457-c34f-4caf-ad34-f70bcc61bc2b"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 19:27:29.083040 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.083032    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cde68457-c34f-4caf-ad34-f70bcc61bc2b-config-out\") pod \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") "
Jul 30 19:27:29.083275 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.083062    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-secret-alertmanager-kube-rbac-proxy\") pod \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") "
Jul 30 19:27:29.083275 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.083093    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-config-volume\") pod \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") "
Jul 30 19:27:29.083275 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.083123    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cde68457-c34f-4caf-ad34-f70bcc61bc2b-alertmanager-main-db\") pod \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") "
Jul 30 19:27:29.083275 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.083171    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\" (UID: \"cde68457-c34f-4caf-ad34-f70bcc61bc2b\") "
Jul 30 19:27:29.083460 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.083286    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cde68457-c34f-4caf-ad34-f70bcc61bc2b-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "cde68457-c34f-4caf-ad34-f70bcc61bc2b" (UID: "cde68457-c34f-4caf-ad34-f70bcc61bc2b"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 19:27:29.083517 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.083473    2554 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cde68457-c34f-4caf-ad34-f70bcc61bc2b-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:27:29.083517 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.083496    2554 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cde68457-c34f-4caf-ad34-f70bcc61bc2b-metrics-client-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:27:29.085065 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.084861    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cde68457-c34f-4caf-ad34-f70bcc61bc2b-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "cde68457-c34f-4caf-ad34-f70bcc61bc2b" (UID: "cde68457-c34f-4caf-ad34-f70bcc61bc2b"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:27:29.086109 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.086067    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cde68457-c34f-4caf-ad34-f70bcc61bc2b-kube-api-access-qnf7r" (OuterVolumeSpecName: "kube-api-access-qnf7r") pod "cde68457-c34f-4caf-ad34-f70bcc61bc2b" (UID: "cde68457-c34f-4caf-ad34-f70bcc61bc2b"). InnerVolumeSpecName "kube-api-access-qnf7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:27:29.086352 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.086303    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "cde68457-c34f-4caf-ad34-f70bcc61bc2b" (UID: "cde68457-c34f-4caf-ad34-f70bcc61bc2b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 19:27:29.086352 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.086340    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "cde68457-c34f-4caf-ad34-f70bcc61bc2b" (UID: "cde68457-c34f-4caf-ad34-f70bcc61bc2b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 19:27:29.086667 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.086626    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cde68457-c34f-4caf-ad34-f70bcc61bc2b-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "cde68457-c34f-4caf-ad34-f70bcc61bc2b" (UID: "cde68457-c34f-4caf-ad34-f70bcc61bc2b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:27:29.086866 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.086841    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "cde68457-c34f-4caf-ad34-f70bcc61bc2b" (UID: "cde68457-c34f-4caf-ad34-f70bcc61bc2b"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 19:27:29.086995 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.086877    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-config-volume" (OuterVolumeSpecName: "config-volume") pod "cde68457-c34f-4caf-ad34-f70bcc61bc2b" (UID: "cde68457-c34f-4caf-ad34-f70bcc61bc2b"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 19:27:29.087752 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.087720    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "cde68457-c34f-4caf-ad34-f70bcc61bc2b" (UID: "cde68457-c34f-4caf-ad34-f70bcc61bc2b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 19:27:29.087850 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.087804    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cde68457-c34f-4caf-ad34-f70bcc61bc2b-config-out" (OuterVolumeSpecName: "config-out") pod "cde68457-c34f-4caf-ad34-f70bcc61bc2b" (UID: "cde68457-c34f-4caf-ad34-f70bcc61bc2b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:27:29.090844 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.090768    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "cde68457-c34f-4caf-ad34-f70bcc61bc2b" (UID: "cde68457-c34f-4caf-ad34-f70bcc61bc2b"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 19:27:29.098412 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.098389    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-web-config" (OuterVolumeSpecName: "web-config") pod "cde68457-c34f-4caf-ad34-f70bcc61bc2b" (UID: "cde68457-c34f-4caf-ad34-f70bcc61bc2b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 19:27:29.184684 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.184638    2554 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cde68457-c34f-4caf-ad34-f70bcc61bc2b-alertmanager-main-db\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:27:29.184684 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.184683    2554 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:27:29.184684 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.184694    2554 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cde68457-c34f-4caf-ad34-f70bcc61bc2b-tls-assets\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:27:29.184684 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.184703    2554 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:27:29.184920 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.184712    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnf7r\" (UniqueName: \"kubernetes.io/projected/cde68457-c34f-4caf-ad34-f70bcc61bc2b-kube-api-access-qnf7r\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:27:29.184920 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.184721    2554 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-secret-alertmanager-main-tls\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:27:29.184920 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.184730    2554 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-cluster-tls-config\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:27:29.184920 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.184738    2554 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-web-config\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:27:29.184920 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.184748    2554 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cde68457-c34f-4caf-ad34-f70bcc61bc2b-config-out\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:27:29.184920 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.184756    2554 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:27:29.184920 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.184763    2554 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cde68457-c34f-4caf-ad34-f70bcc61bc2b-config-volume\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:27:29.398596 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.398516    2554 generic.go:356] "Generic (PLEG): container finished" podID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerID="0c311d55e20bc96d15126a632cf106cfb17fa1426cf2043b7512fc5b1d5cfd85" exitCode=0
Jul 30 19:27:29.398596 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.398546    2554 generic.go:356] "Generic (PLEG): container finished" podID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerID="c998a524cafe1ccd424e7162b0d8f7a64161975bb4a33bd94fbeae506f461975" exitCode=0
Jul 30 19:27:29.398766 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.398607    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.398766 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.398605    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cde68457-c34f-4caf-ad34-f70bcc61bc2b","Type":"ContainerDied","Data":"0c311d55e20bc96d15126a632cf106cfb17fa1426cf2043b7512fc5b1d5cfd85"}
Jul 30 19:27:29.398766 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.398683    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cde68457-c34f-4caf-ad34-f70bcc61bc2b","Type":"ContainerDied","Data":"c998a524cafe1ccd424e7162b0d8f7a64161975bb4a33bd94fbeae506f461975"}
Jul 30 19:27:29.398766 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.398719    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cde68457-c34f-4caf-ad34-f70bcc61bc2b","Type":"ContainerDied","Data":"a87da962cee60f56feda392e031af26a21d7cf0ddc676d00e6e7d6e43c68084f"}
Jul 30 19:27:29.398766 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.398761    2554 scope.go:117] "RemoveContainer" containerID="d786666e53ed2cf1642c41e661dfc6fafafb3d5f5506304c1f230a49d5bc41e3"
Jul 30 19:27:29.411602 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.411580    2554 scope.go:117] "RemoveContainer" containerID="0c311d55e20bc96d15126a632cf106cfb17fa1426cf2043b7512fc5b1d5cfd85"
Jul 30 19:27:29.418881 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.418861    2554 scope.go:117] "RemoveContainer" containerID="a980a8c3908293089d8c9815722a2daff74ced959e3a29a0236c450bff24ca8f"
Jul 30 19:27:29.423396 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.423356    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 19:27:29.426627 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.426606    2554 scope.go:117] "RemoveContainer" containerID="c998a524cafe1ccd424e7162b0d8f7a64161975bb4a33bd94fbeae506f461975"
Jul 30 19:27:29.427971 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.427927    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 19:27:29.433550 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.433531    2554 scope.go:117] "RemoveContainer" containerID="b1aff326ca513262e2baf9f1056d7d73c2280ada8a16e3c87a4b7731fabeb5a2"
Jul 30 19:27:29.440713 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.440693    2554 scope.go:117] "RemoveContainer" containerID="f412e1c4a70c8c483c5761b331e7cd6e9c6ea41c66a5d3a65aed5b12c2457af7"
Jul 30 19:27:29.447924 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.447904    2554 scope.go:117] "RemoveContainer" containerID="7398ad818904879a9bf2ec6f959cea4cebdb68af664cc686a7ac60517dfb21b5"
Jul 30 19:27:29.454762 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.454738    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 19:27:29.455096 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455077    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerName="kube-rbac-proxy"
Jul 30 19:27:29.455141 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455098    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerName="kube-rbac-proxy"
Jul 30 19:27:29.455141 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455111    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerName="init-config-reloader"
Jul 30 19:27:29.455141 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455117    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerName="init-config-reloader"
Jul 30 19:27:29.455141 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455131    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerName="prom-label-proxy"
Jul 30 19:27:29.455141 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455139    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerName="prom-label-proxy"
Jul 30 19:27:29.455343 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455145    2554 scope.go:117] "RemoveContainer" containerID="d786666e53ed2cf1642c41e661dfc6fafafb3d5f5506304c1f230a49d5bc41e3"
Jul 30 19:27:29.455343 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455153    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf" containerName="registry"
Jul 30 19:27:29.455343 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455163    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf" containerName="registry"
Jul 30 19:27:29.455343 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455175    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerName="alertmanager"
Jul 30 19:27:29.455343 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455183    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerName="alertmanager"
Jul 30 19:27:29.455343 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455191    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerName="config-reloader"
Jul 30 19:27:29.455343 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455198    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerName="config-reloader"
Jul 30 19:27:29.455343 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455205    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerName="kube-rbac-proxy-web"
Jul 30 19:27:29.455343 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455212    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerName="kube-rbac-proxy-web"
Jul 30 19:27:29.455343 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455219    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa052cb1-c5e2-465a-89f2-a53c960e51fb" containerName="console"
Jul 30 19:27:29.455343 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455225    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa052cb1-c5e2-465a-89f2-a53c960e51fb" containerName="console"
Jul 30 19:27:29.455343 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455235    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerName="kube-rbac-proxy-metric"
Jul 30 19:27:29.455343 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455241    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerName="kube-rbac-proxy-metric"
Jul 30 19:27:29.455343 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455299    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerName="config-reloader"
Jul 30 19:27:29.455343 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455313    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerName="kube-rbac-proxy-metric"
Jul 30 19:27:29.455343 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455322    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerName="kube-rbac-proxy"
Jul 30 19:27:29.455343 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455334    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerName="prom-label-proxy"
Jul 30 19:27:29.455343 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455345    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdbedcab-2bf0-4b28-abe0-9ccbeadfddcf" containerName="registry"
Jul 30 19:27:29.455343 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455351    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerName="alertmanager"
Jul 30 19:27:29.455343 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455356    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" containerName="kube-rbac-proxy-web"
Jul 30 19:27:29.455979 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455361    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa052cb1-c5e2-465a-89f2-a53c960e51fb" containerName="console"
Jul 30 19:27:29.455979 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:27:29.455443    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d786666e53ed2cf1642c41e661dfc6fafafb3d5f5506304c1f230a49d5bc41e3\": container with ID starting with d786666e53ed2cf1642c41e661dfc6fafafb3d5f5506304c1f230a49d5bc41e3 not found: ID does not exist" containerID="d786666e53ed2cf1642c41e661dfc6fafafb3d5f5506304c1f230a49d5bc41e3"
Jul 30 19:27:29.455979 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455468    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d786666e53ed2cf1642c41e661dfc6fafafb3d5f5506304c1f230a49d5bc41e3"} err="failed to get container status \"d786666e53ed2cf1642c41e661dfc6fafafb3d5f5506304c1f230a49d5bc41e3\": rpc error: code = NotFound desc = could not find container \"d786666e53ed2cf1642c41e661dfc6fafafb3d5f5506304c1f230a49d5bc41e3\": container with ID starting with d786666e53ed2cf1642c41e661dfc6fafafb3d5f5506304c1f230a49d5bc41e3 not found: ID does not exist"
Jul 30 19:27:29.455979 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455487    2554 scope.go:117] "RemoveContainer" containerID="0c311d55e20bc96d15126a632cf106cfb17fa1426cf2043b7512fc5b1d5cfd85"
Jul 30 19:27:29.455979 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:27:29.455725    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c311d55e20bc96d15126a632cf106cfb17fa1426cf2043b7512fc5b1d5cfd85\": container with ID starting with 0c311d55e20bc96d15126a632cf106cfb17fa1426cf2043b7512fc5b1d5cfd85 not found: ID does not exist" containerID="0c311d55e20bc96d15126a632cf106cfb17fa1426cf2043b7512fc5b1d5cfd85"
Jul 30 19:27:29.455979 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455742    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c311d55e20bc96d15126a632cf106cfb17fa1426cf2043b7512fc5b1d5cfd85"} err="failed to get container status \"0c311d55e20bc96d15126a632cf106cfb17fa1426cf2043b7512fc5b1d5cfd85\": rpc error: code = NotFound desc = could not find container \"0c311d55e20bc96d15126a632cf106cfb17fa1426cf2043b7512fc5b1d5cfd85\": container with ID starting with 0c311d55e20bc96d15126a632cf106cfb17fa1426cf2043b7512fc5b1d5cfd85 not found: ID does not exist"
Jul 30 19:27:29.455979 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.455754    2554 scope.go:117] "RemoveContainer" containerID="a980a8c3908293089d8c9815722a2daff74ced959e3a29a0236c450bff24ca8f"
Jul 30 19:27:29.456294 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:27:29.455986    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a980a8c3908293089d8c9815722a2daff74ced959e3a29a0236c450bff24ca8f\": container with ID starting with a980a8c3908293089d8c9815722a2daff74ced959e3a29a0236c450bff24ca8f not found: ID does not exist" containerID="a980a8c3908293089d8c9815722a2daff74ced959e3a29a0236c450bff24ca8f"
Jul 30 19:27:29.456294 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.456010    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a980a8c3908293089d8c9815722a2daff74ced959e3a29a0236c450bff24ca8f"} err="failed to get container status \"a980a8c3908293089d8c9815722a2daff74ced959e3a29a0236c450bff24ca8f\": rpc error: code = NotFound desc = could not find container \"a980a8c3908293089d8c9815722a2daff74ced959e3a29a0236c450bff24ca8f\": container with ID starting with a980a8c3908293089d8c9815722a2daff74ced959e3a29a0236c450bff24ca8f not found: ID does not exist"
Jul 30 19:27:29.456294 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.456028    2554 scope.go:117] "RemoveContainer" containerID="c998a524cafe1ccd424e7162b0d8f7a64161975bb4a33bd94fbeae506f461975"
Jul 30 19:27:29.456294 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:27:29.456256    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c998a524cafe1ccd424e7162b0d8f7a64161975bb4a33bd94fbeae506f461975\": container with ID starting with c998a524cafe1ccd424e7162b0d8f7a64161975bb4a33bd94fbeae506f461975 not found: ID does not exist" containerID="c998a524cafe1ccd424e7162b0d8f7a64161975bb4a33bd94fbeae506f461975"
Jul 30 19:27:29.456294 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.456278    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c998a524cafe1ccd424e7162b0d8f7a64161975bb4a33bd94fbeae506f461975"} err="failed to get container status \"c998a524cafe1ccd424e7162b0d8f7a64161975bb4a33bd94fbeae506f461975\": rpc error: code = NotFound desc = could not find container \"c998a524cafe1ccd424e7162b0d8f7a64161975bb4a33bd94fbeae506f461975\": container with ID starting with c998a524cafe1ccd424e7162b0d8f7a64161975bb4a33bd94fbeae506f461975 not found: ID does not exist"
Jul 30 19:27:29.456294 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.456295    2554 scope.go:117] "RemoveContainer" containerID="b1aff326ca513262e2baf9f1056d7d73c2280ada8a16e3c87a4b7731fabeb5a2"
Jul 30 19:27:29.456535 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:27:29.456508    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1aff326ca513262e2baf9f1056d7d73c2280ada8a16e3c87a4b7731fabeb5a2\": container with ID starting with b1aff326ca513262e2baf9f1056d7d73c2280ada8a16e3c87a4b7731fabeb5a2 not found: ID does not exist" containerID="b1aff326ca513262e2baf9f1056d7d73c2280ada8a16e3c87a4b7731fabeb5a2"
Jul 30 19:27:29.456535 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.456527    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1aff326ca513262e2baf9f1056d7d73c2280ada8a16e3c87a4b7731fabeb5a2"} err="failed to get container status \"b1aff326ca513262e2baf9f1056d7d73c2280ada8a16e3c87a4b7731fabeb5a2\": rpc error: code = NotFound desc = could not find container \"b1aff326ca513262e2baf9f1056d7d73c2280ada8a16e3c87a4b7731fabeb5a2\": container with ID starting with b1aff326ca513262e2baf9f1056d7d73c2280ada8a16e3c87a4b7731fabeb5a2 not found: ID does not exist"
Jul 30 19:27:29.456613 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.456538    2554 scope.go:117] "RemoveContainer" containerID="f412e1c4a70c8c483c5761b331e7cd6e9c6ea41c66a5d3a65aed5b12c2457af7"
Jul 30 19:27:29.456752 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:27:29.456737    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f412e1c4a70c8c483c5761b331e7cd6e9c6ea41c66a5d3a65aed5b12c2457af7\": container with ID starting with f412e1c4a70c8c483c5761b331e7cd6e9c6ea41c66a5d3a65aed5b12c2457af7 not found: ID does not exist" containerID="f412e1c4a70c8c483c5761b331e7cd6e9c6ea41c66a5d3a65aed5b12c2457af7"
Jul 30 19:27:29.456787 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.456758    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f412e1c4a70c8c483c5761b331e7cd6e9c6ea41c66a5d3a65aed5b12c2457af7"} err="failed to get container status \"f412e1c4a70c8c483c5761b331e7cd6e9c6ea41c66a5d3a65aed5b12c2457af7\": rpc error: code = NotFound desc = could not find container \"f412e1c4a70c8c483c5761b331e7cd6e9c6ea41c66a5d3a65aed5b12c2457af7\": container with ID starting with f412e1c4a70c8c483c5761b331e7cd6e9c6ea41c66a5d3a65aed5b12c2457af7 not found: ID does not exist"
Jul 30 19:27:29.456787 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.456769    2554 scope.go:117] "RemoveContainer" containerID="7398ad818904879a9bf2ec6f959cea4cebdb68af664cc686a7ac60517dfb21b5"
Jul 30 19:27:29.456991 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:27:29.456976    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7398ad818904879a9bf2ec6f959cea4cebdb68af664cc686a7ac60517dfb21b5\": container with ID starting with 7398ad818904879a9bf2ec6f959cea4cebdb68af664cc686a7ac60517dfb21b5 not found: ID does not exist" containerID="7398ad818904879a9bf2ec6f959cea4cebdb68af664cc686a7ac60517dfb21b5"
Jul 30 19:27:29.457041 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.456993    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7398ad818904879a9bf2ec6f959cea4cebdb68af664cc686a7ac60517dfb21b5"} err="failed to get container status \"7398ad818904879a9bf2ec6f959cea4cebdb68af664cc686a7ac60517dfb21b5\": rpc error: code = NotFound desc = could not find container \"7398ad818904879a9bf2ec6f959cea4cebdb68af664cc686a7ac60517dfb21b5\": container with ID starting with 7398ad818904879a9bf2ec6f959cea4cebdb68af664cc686a7ac60517dfb21b5 not found: ID does not exist"
Jul 30 19:27:29.457041 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.457004    2554 scope.go:117] "RemoveContainer" containerID="d786666e53ed2cf1642c41e661dfc6fafafb3d5f5506304c1f230a49d5bc41e3"
Jul 30 19:27:29.457221 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.457202    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d786666e53ed2cf1642c41e661dfc6fafafb3d5f5506304c1f230a49d5bc41e3"} err="failed to get container status \"d786666e53ed2cf1642c41e661dfc6fafafb3d5f5506304c1f230a49d5bc41e3\": rpc error: code = NotFound desc = could not find container \"d786666e53ed2cf1642c41e661dfc6fafafb3d5f5506304c1f230a49d5bc41e3\": container with ID starting with d786666e53ed2cf1642c41e661dfc6fafafb3d5f5506304c1f230a49d5bc41e3 not found: ID does not exist"
Jul 30 19:27:29.457271 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.457221    2554 scope.go:117] "RemoveContainer" containerID="0c311d55e20bc96d15126a632cf106cfb17fa1426cf2043b7512fc5b1d5cfd85"
Jul 30 19:27:29.457439 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.457420    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c311d55e20bc96d15126a632cf106cfb17fa1426cf2043b7512fc5b1d5cfd85"} err="failed to get container status \"0c311d55e20bc96d15126a632cf106cfb17fa1426cf2043b7512fc5b1d5cfd85\": rpc error: code = NotFound desc = could not find container \"0c311d55e20bc96d15126a632cf106cfb17fa1426cf2043b7512fc5b1d5cfd85\": container with ID starting with 0c311d55e20bc96d15126a632cf106cfb17fa1426cf2043b7512fc5b1d5cfd85 not found: ID does not exist"
Jul 30 19:27:29.457491 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.457441    2554 scope.go:117] "RemoveContainer" containerID="a980a8c3908293089d8c9815722a2daff74ced959e3a29a0236c450bff24ca8f"
Jul 30 19:27:29.457662 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.457645    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a980a8c3908293089d8c9815722a2daff74ced959e3a29a0236c450bff24ca8f"} err="failed to get container status \"a980a8c3908293089d8c9815722a2daff74ced959e3a29a0236c450bff24ca8f\": rpc error: code = NotFound desc = could not find container \"a980a8c3908293089d8c9815722a2daff74ced959e3a29a0236c450bff24ca8f\": container with ID starting with a980a8c3908293089d8c9815722a2daff74ced959e3a29a0236c450bff24ca8f not found: ID does not exist"
Jul 30 19:27:29.457662 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.457662    2554 scope.go:117] "RemoveContainer" containerID="c998a524cafe1ccd424e7162b0d8f7a64161975bb4a33bd94fbeae506f461975"
Jul 30 19:27:29.457885 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.457866    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c998a524cafe1ccd424e7162b0d8f7a64161975bb4a33bd94fbeae506f461975"} err="failed to get container status \"c998a524cafe1ccd424e7162b0d8f7a64161975bb4a33bd94fbeae506f461975\": rpc error: code = NotFound desc = could not find container \"c998a524cafe1ccd424e7162b0d8f7a64161975bb4a33bd94fbeae506f461975\": container with ID starting with c998a524cafe1ccd424e7162b0d8f7a64161975bb4a33bd94fbeae506f461975 not found: ID does not exist"
Jul 30 19:27:29.458020 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.457888    2554 scope.go:117] "RemoveContainer" containerID="b1aff326ca513262e2baf9f1056d7d73c2280ada8a16e3c87a4b7731fabeb5a2"
Jul 30 19:27:29.458133 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.458116    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1aff326ca513262e2baf9f1056d7d73c2280ada8a16e3c87a4b7731fabeb5a2"} err="failed to get container status \"b1aff326ca513262e2baf9f1056d7d73c2280ada8a16e3c87a4b7731fabeb5a2\": rpc error: code = NotFound desc = could not find container \"b1aff326ca513262e2baf9f1056d7d73c2280ada8a16e3c87a4b7731fabeb5a2\": container with ID starting with b1aff326ca513262e2baf9f1056d7d73c2280ada8a16e3c87a4b7731fabeb5a2 not found: ID does not exist"
Jul 30 19:27:29.458182 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.458133    2554 scope.go:117] "RemoveContainer" containerID="f412e1c4a70c8c483c5761b331e7cd6e9c6ea41c66a5d3a65aed5b12c2457af7"
Jul 30 19:27:29.458300 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.458284    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f412e1c4a70c8c483c5761b331e7cd6e9c6ea41c66a5d3a65aed5b12c2457af7"} err="failed to get container status \"f412e1c4a70c8c483c5761b331e7cd6e9c6ea41c66a5d3a65aed5b12c2457af7\": rpc error: code = NotFound desc = could not find container \"f412e1c4a70c8c483c5761b331e7cd6e9c6ea41c66a5d3a65aed5b12c2457af7\": container with ID starting with f412e1c4a70c8c483c5761b331e7cd6e9c6ea41c66a5d3a65aed5b12c2457af7 not found: ID does not exist"
Jul 30 19:27:29.458348 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.458300    2554 scope.go:117] "RemoveContainer" containerID="7398ad818904879a9bf2ec6f959cea4cebdb68af664cc686a7ac60517dfb21b5"
Jul 30 19:27:29.458494 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.458474    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7398ad818904879a9bf2ec6f959cea4cebdb68af664cc686a7ac60517dfb21b5"} err="failed to get container status \"7398ad818904879a9bf2ec6f959cea4cebdb68af664cc686a7ac60517dfb21b5\": rpc error: code = NotFound desc = could not find container \"7398ad818904879a9bf2ec6f959cea4cebdb68af664cc686a7ac60517dfb21b5\": container with ID starting with 7398ad818904879a9bf2ec6f959cea4cebdb68af664cc686a7ac60517dfb21b5 not found: ID does not exist"
Jul 30 19:27:29.459020 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.459005    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.461521 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.461498    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 30 19:27:29.461624 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.461498    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 30 19:27:29.461681 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.461647    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-xm54g\""
Jul 30 19:27:29.461721 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.461647    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 30 19:27:29.461769 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.461752    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 30 19:27:29.461983 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.461964    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 30 19:27:29.462075 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.461991    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 30 19:27:29.462075 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.462006    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 30 19:27:29.462193 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.462179    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 30 19:27:29.467012 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.466993    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 30 19:27:29.475779 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.475759    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 19:27:29.588024 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.587939    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d47a719-7ccd-4b74-9012-201963f6a7ff-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.588210 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.588064    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5d47a719-7ccd-4b74-9012-201963f6a7ff-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.588210 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.588115    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5d47a719-7ccd-4b74-9012-201963f6a7ff-config-volume\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.588210 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.588136    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5d47a719-7ccd-4b74-9012-201963f6a7ff-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.588210 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.588152    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5d47a719-7ccd-4b74-9012-201963f6a7ff-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.588210 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.588169    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d47a719-7ccd-4b74-9012-201963f6a7ff-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.588406 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.588216    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5d47a719-7ccd-4b74-9012-201963f6a7ff-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.588406 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.588245    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5d47a719-7ccd-4b74-9012-201963f6a7ff-config-out\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.588406 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.588284    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d47a719-7ccd-4b74-9012-201963f6a7ff-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.588406 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.588305    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5d47a719-7ccd-4b74-9012-201963f6a7ff-web-config\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.588406 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.588335    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qkns\" (UniqueName: \"kubernetes.io/projected/5d47a719-7ccd-4b74-9012-201963f6a7ff-kube-api-access-7qkns\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.588406 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.588365    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d47a719-7ccd-4b74-9012-201963f6a7ff-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.588406 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.588385    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5d47a719-7ccd-4b74-9012-201963f6a7ff-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.689040 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.688922    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5d47a719-7ccd-4b74-9012-201963f6a7ff-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.689204 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.689071    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5d47a719-7ccd-4b74-9012-201963f6a7ff-config-volume\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.689204 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.689094    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5d47a719-7ccd-4b74-9012-201963f6a7ff-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.689204 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.689114    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5d47a719-7ccd-4b74-9012-201963f6a7ff-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.689204 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.689132    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d47a719-7ccd-4b74-9012-201963f6a7ff-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.689204 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.689156    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5d47a719-7ccd-4b74-9012-201963f6a7ff-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.689204 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.689173    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5d47a719-7ccd-4b74-9012-201963f6a7ff-config-out\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.689542 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.689321    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d47a719-7ccd-4b74-9012-201963f6a7ff-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.689542 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.689362    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5d47a719-7ccd-4b74-9012-201963f6a7ff-web-config\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.689542 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.689422    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qkns\" (UniqueName: \"kubernetes.io/projected/5d47a719-7ccd-4b74-9012-201963f6a7ff-kube-api-access-7qkns\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.689542 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.689457    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d47a719-7ccd-4b74-9012-201963f6a7ff-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.689542 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.689492    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5d47a719-7ccd-4b74-9012-201963f6a7ff-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.689799 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.689549    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d47a719-7ccd-4b74-9012-201963f6a7ff-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.689799 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.689647    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5d47a719-7ccd-4b74-9012-201963f6a7ff-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.690243 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.690118    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d47a719-7ccd-4b74-9012-201963f6a7ff-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.690243 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.690237    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d47a719-7ccd-4b74-9012-201963f6a7ff-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.691871 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.691842    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5d47a719-7ccd-4b74-9012-201963f6a7ff-config-out\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.692162 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.692136    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5d47a719-7ccd-4b74-9012-201963f6a7ff-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.692260 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.692224    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5d47a719-7ccd-4b74-9012-201963f6a7ff-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.692472 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.692454    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5d47a719-7ccd-4b74-9012-201963f6a7ff-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.692688 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.692668    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d47a719-7ccd-4b74-9012-201963f6a7ff-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.692908 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.692882    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5d47a719-7ccd-4b74-9012-201963f6a7ff-web-config\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.692908 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.692898    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5d47a719-7ccd-4b74-9012-201963f6a7ff-config-volume\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.693051 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.693013    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d47a719-7ccd-4b74-9012-201963f6a7ff-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.693878 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.693861    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5d47a719-7ccd-4b74-9012-201963f6a7ff-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.696573 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.696555    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qkns\" (UniqueName: \"kubernetes.io/projected/5d47a719-7ccd-4b74-9012-201963f6a7ff-kube-api-access-7qkns\") pod \"alertmanager-main-0\" (UID: \"5d47a719-7ccd-4b74-9012-201963f6a7ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.769670 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.769627    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 30 19:27:29.907182 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.907110    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 30 19:27:29.910564 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:27:29.910534    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d47a719_7ccd_4b74_9012_201963f6a7ff.slice/crio-2797d99ae08ef0d3eddf84fad4d35d966343dafdd795ac7a080da4f3ee0fe392 WatchSource:0}: Error finding container 2797d99ae08ef0d3eddf84fad4d35d966343dafdd795ac7a080da4f3ee0fe392: Status 404 returned error can't find the container with id 2797d99ae08ef0d3eddf84fad4d35d966343dafdd795ac7a080da4f3ee0fe392
Jul 30 19:27:29.979865 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:29.979830    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:27:30.403873 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:30.403839    2554 generic.go:356] "Generic (PLEG): container finished" podID="5d47a719-7ccd-4b74-9012-201963f6a7ff" containerID="8c232aca98ee39f54384ecb47ae91ffa82521258b2e2d54a77e6296c0bdc5955" exitCode=0
Jul 30 19:27:30.404336 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:30.403939    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d47a719-7ccd-4b74-9012-201963f6a7ff","Type":"ContainerDied","Data":"8c232aca98ee39f54384ecb47ae91ffa82521258b2e2d54a77e6296c0bdc5955"}
Jul 30 19:27:30.404336 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:30.404043    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d47a719-7ccd-4b74-9012-201963f6a7ff","Type":"ContainerStarted","Data":"2797d99ae08ef0d3eddf84fad4d35d966343dafdd795ac7a080da4f3ee0fe392"}
Jul 30 19:27:30.485649 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:30.485622    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cde68457-c34f-4caf-ad34-f70bcc61bc2b" path="/var/lib/kubelet/pods/cde68457-c34f-4caf-ad34-f70bcc61bc2b/volumes"
Jul 30 19:27:31.412876 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:31.412841    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d47a719-7ccd-4b74-9012-201963f6a7ff","Type":"ContainerStarted","Data":"08fe4d40e0a71763466df438f14622774b2b8e9c7fe6426feb179f59d52d1485"}
Jul 30 19:27:31.413264 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:31.412933    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d47a719-7ccd-4b74-9012-201963f6a7ff","Type":"ContainerStarted","Data":"42fe0c87302309ffed9ad7df5737bb0e561fa6581250bff9fae2eb5b5da312d4"}
Jul 30 19:27:31.413264 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:31.412977    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d47a719-7ccd-4b74-9012-201963f6a7ff","Type":"ContainerStarted","Data":"8247aa30d3b47f9dee05e3907e20d7211d3ac8feaf97e0bbc9a18bca30de1b58"}
Jul 30 19:27:31.413264 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:31.413006    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d47a719-7ccd-4b74-9012-201963f6a7ff","Type":"ContainerStarted","Data":"26c5554a0825ec99c6ea8056c19649df52fb55e01770b7dd142affa2080ce172"}
Jul 30 19:27:31.413264 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:31.413030    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d47a719-7ccd-4b74-9012-201963f6a7ff","Type":"ContainerStarted","Data":"cf1de749ef4fc4d7ab920d9f8c670c4b7ba47e9557950976c13c8440b9ca0442"}
Jul 30 19:27:31.413264 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:31.413052    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5d47a719-7ccd-4b74-9012-201963f6a7ff","Type":"ContainerStarted","Data":"52ff717803bcbc46582b12ba5554277362214b63826810ae022ab80ff73a052a"}
Jul 30 19:27:31.438797 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:31.438741    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.42307103 podStartE2EDuration="2.438722752s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.651722ms" isStatefulPod=false podCreationTimestamp="2026-07-30 19:27:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:27:31.4385354 +0000 UTC m=+179.403116482" watchObservedRunningTime="2026-07-30 19:27:31.438722752 +0000 UTC m=+179.403303835"
Jul 30 19:27:35.425947 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:35.425912    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:27:35.484477 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:35.484437    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:27:41.375536 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.375493    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-869fdf5f7b-h9x47"]
Jul 30 19:27:41.378878 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.378855    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:27:41.381704 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.381677    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 30 19:27:41.381704 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.381705    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 30 19:27:41.382000 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.381730    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 30 19:27:41.382471 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.382454    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 30 19:27:41.382736 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.382711    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 30 19:27:41.382736 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.382713    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-smf57\""
Jul 30 19:27:41.382867 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.382766    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 30 19:27:41.382932 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.382866    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 30 19:27:41.387267 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.387246    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 30 19:27:41.391926 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.391897    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-869fdf5f7b-h9x47"]
Jul 30 19:27:41.397367 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.397342    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:27:41.490703 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.490669    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/539b3bf5-a852-4029-8107-2d89ee7d6e5b-console-oauth-config\") pod \"console-869fdf5f7b-h9x47\" (UID: \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\") " pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:27:41.490703 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.490707    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/539b3bf5-a852-4029-8107-2d89ee7d6e5b-console-config\") pod \"console-869fdf5f7b-h9x47\" (UID: \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\") " pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:27:41.490911 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.490726    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/539b3bf5-a852-4029-8107-2d89ee7d6e5b-trusted-ca-bundle\") pod \"console-869fdf5f7b-h9x47\" (UID: \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\") " pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:27:41.490911 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.490780    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/539b3bf5-a852-4029-8107-2d89ee7d6e5b-console-serving-cert\") pod \"console-869fdf5f7b-h9x47\" (UID: \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\") " pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:27:41.490911 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.490822    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtfqx\" (UniqueName: \"kubernetes.io/projected/539b3bf5-a852-4029-8107-2d89ee7d6e5b-kube-api-access-dtfqx\") pod \"console-869fdf5f7b-h9x47\" (UID: \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\") " pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:27:41.490911 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.490884    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/539b3bf5-a852-4029-8107-2d89ee7d6e5b-oauth-serving-cert\") pod \"console-869fdf5f7b-h9x47\" (UID: \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\") " pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:27:41.491057 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.491005    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/539b3bf5-a852-4029-8107-2d89ee7d6e5b-service-ca\") pod \"console-869fdf5f7b-h9x47\" (UID: \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\") " pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:27:41.499237 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.499212    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:27:41.591453 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.591412    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/539b3bf5-a852-4029-8107-2d89ee7d6e5b-oauth-serving-cert\") pod \"console-869fdf5f7b-h9x47\" (UID: \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\") " pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:27:41.591629 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.591470    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/539b3bf5-a852-4029-8107-2d89ee7d6e5b-service-ca\") pod \"console-869fdf5f7b-h9x47\" (UID: \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\") " pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:27:41.591629 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.591493    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/539b3bf5-a852-4029-8107-2d89ee7d6e5b-console-oauth-config\") pod \"console-869fdf5f7b-h9x47\" (UID: \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\") " pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:27:41.591629 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.591511    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/539b3bf5-a852-4029-8107-2d89ee7d6e5b-console-config\") pod \"console-869fdf5f7b-h9x47\" (UID: \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\") " pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:27:41.591629 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.591527    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/539b3bf5-a852-4029-8107-2d89ee7d6e5b-trusted-ca-bundle\") pod \"console-869fdf5f7b-h9x47\" (UID: \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\") " pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:27:41.591629 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.591550    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/539b3bf5-a852-4029-8107-2d89ee7d6e5b-console-serving-cert\") pod \"console-869fdf5f7b-h9x47\" (UID: \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\") " pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:27:41.591629 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.591579    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtfqx\" (UniqueName: \"kubernetes.io/projected/539b3bf5-a852-4029-8107-2d89ee7d6e5b-kube-api-access-dtfqx\") pod \"console-869fdf5f7b-h9x47\" (UID: \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\") " pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:27:41.592393 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.592354    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/539b3bf5-a852-4029-8107-2d89ee7d6e5b-oauth-serving-cert\") pod \"console-869fdf5f7b-h9x47\" (UID: \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\") " pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:27:41.592511 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.592359    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/539b3bf5-a852-4029-8107-2d89ee7d6e5b-console-config\") pod \"console-869fdf5f7b-h9x47\" (UID: \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\") " pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:27:41.592511 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.592416    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/539b3bf5-a852-4029-8107-2d89ee7d6e5b-service-ca\") pod \"console-869fdf5f7b-h9x47\" (UID: \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\") " pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:27:41.592599 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.592522    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/539b3bf5-a852-4029-8107-2d89ee7d6e5b-trusted-ca-bundle\") pod \"console-869fdf5f7b-h9x47\" (UID: \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\") " pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:27:41.594048 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.594031    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/539b3bf5-a852-4029-8107-2d89ee7d6e5b-console-oauth-config\") pod \"console-869fdf5f7b-h9x47\" (UID: \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\") " pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:27:41.594326 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.594302    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/539b3bf5-a852-4029-8107-2d89ee7d6e5b-console-serving-cert\") pod \"console-869fdf5f7b-h9x47\" (UID: \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\") " pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:27:41.598793 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.598760    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:27:41.600859 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.600841    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtfqx\" (UniqueName: \"kubernetes.io/projected/539b3bf5-a852-4029-8107-2d89ee7d6e5b-kube-api-access-dtfqx\") pod \"console-869fdf5f7b-h9x47\" (UID: \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\") " pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:27:41.689972 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.689852    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:27:41.821431 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:41.821390    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-869fdf5f7b-h9x47"]
Jul 30 19:27:41.824600 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:27:41.824563    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod539b3bf5_a852_4029_8107_2d89ee7d6e5b.slice/crio-34e8cef42688fed453fa7c56a4a4397c36f9ce0e81177ad5c00e72fe9c35a63b WatchSource:0}: Error finding container 34e8cef42688fed453fa7c56a4a4397c36f9ce0e81177ad5c00e72fe9c35a63b: Status 404 returned error can't find the container with id 34e8cef42688fed453fa7c56a4a4397c36f9ce0e81177ad5c00e72fe9c35a63b
Jul 30 19:27:42.458935 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:42.458898    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-869fdf5f7b-h9x47" event={"ID":"539b3bf5-a852-4029-8107-2d89ee7d6e5b","Type":"ContainerStarted","Data":"82adce2724a2f1c30bd318873c36ef2e4146fcce1295deaf1dffee7ae526e661"}
Jul 30 19:27:42.458935 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:42.458968    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-869fdf5f7b-h9x47" event={"ID":"539b3bf5-a852-4029-8107-2d89ee7d6e5b","Type":"ContainerStarted","Data":"34e8cef42688fed453fa7c56a4a4397c36f9ce0e81177ad5c00e72fe9c35a63b"}
Jul 30 19:27:42.476122 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:42.476076    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-869fdf5f7b-h9x47" podStartSLOduration=1.47606015 podStartE2EDuration="1.47606015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:27:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:27:42.475790504 +0000 UTC m=+190.440371611" watchObservedRunningTime="2026-07-30 19:27:42.47606015 +0000 UTC m=+190.440641233"
Jul 30 19:27:51.690470 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:51.690426    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:27:51.690867 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:51.690509    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:27:51.695374 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:51.695353    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:27:52.502423 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:52.502388    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:27:52.664502 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:52.664468    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:27:52.720001 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:52.719935    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:27:52.776251 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:27:52.776158    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:28:15.969352 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:28:15.969304    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:28:16.018591 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:28:16.018561    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:28:47.141919 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:28:47.141880    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:28:47.238511 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:28:47.238476    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:28:47.787445 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:28:47.787409    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc"]
Jul 30 19:28:47.792004 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:28:47.791977    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc"
Jul 30 19:28:47.794433 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:28:47.794408    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 30 19:28:47.794433 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:28:47.794411    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 30 19:28:47.794606 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:28:47.794530    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-m4qtq\""
Jul 30 19:28:47.800081 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:28:47.800057    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc"]
Jul 30 19:28:47.970212 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:28:47.970172    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktq2p\" (UniqueName: \"kubernetes.io/projected/e507605f-3466-4993-bbe9-d4a0598f4768-kube-api-access-ktq2p\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc\" (UID: \"e507605f-3466-4993-bbe9-d4a0598f4768\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc"
Jul 30 19:28:47.970212 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:28:47.970218    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e507605f-3466-4993-bbe9-d4a0598f4768-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc\" (UID: \"e507605f-3466-4993-bbe9-d4a0598f4768\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc"
Jul 30 19:28:47.970437 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:28:47.970242    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e507605f-3466-4993-bbe9-d4a0598f4768-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc\" (UID: \"e507605f-3466-4993-bbe9-d4a0598f4768\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc"
Jul 30 19:28:48.071371 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:28:48.071284    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktq2p\" (UniqueName: \"kubernetes.io/projected/e507605f-3466-4993-bbe9-d4a0598f4768-kube-api-access-ktq2p\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc\" (UID: \"e507605f-3466-4993-bbe9-d4a0598f4768\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc"
Jul 30 19:28:48.071371 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:28:48.071334    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e507605f-3466-4993-bbe9-d4a0598f4768-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc\" (UID: \"e507605f-3466-4993-bbe9-d4a0598f4768\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc"
Jul 30 19:28:48.071566 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:28:48.071356    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e507605f-3466-4993-bbe9-d4a0598f4768-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc\" (UID: \"e507605f-3466-4993-bbe9-d4a0598f4768\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc"
Jul 30 19:28:48.071767 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:28:48.071748    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e507605f-3466-4993-bbe9-d4a0598f4768-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc\" (UID: \"e507605f-3466-4993-bbe9-d4a0598f4768\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc"
Jul 30 19:28:48.071807 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:28:48.071762    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e507605f-3466-4993-bbe9-d4a0598f4768-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc\" (UID: \"e507605f-3466-4993-bbe9-d4a0598f4768\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc"
Jul 30 19:28:48.081448 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:28:48.081417    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktq2p\" (UniqueName: \"kubernetes.io/projected/e507605f-3466-4993-bbe9-d4a0598f4768-kube-api-access-ktq2p\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc\" (UID: \"e507605f-3466-4993-bbe9-d4a0598f4768\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc"
Jul 30 19:28:48.102259 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:28:48.102225    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc"
Jul 30 19:28:48.233301 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:28:48.233275    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc"]
Jul 30 19:28:48.237216 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:28:48.237181    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode507605f_3466_4993_bbe9_d4a0598f4768.slice/crio-9b3494d030ce0c783fa3b4785aa296e57e4f75dcc47ddd9fddf883e73e59ad19 WatchSource:0}: Error finding container 9b3494d030ce0c783fa3b4785aa296e57e4f75dcc47ddd9fddf883e73e59ad19: Status 404 returned error can't find the container with id 9b3494d030ce0c783fa3b4785aa296e57e4f75dcc47ddd9fddf883e73e59ad19
Jul 30 19:28:48.719802 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:28:48.719769    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc" event={"ID":"e507605f-3466-4993-bbe9-d4a0598f4768","Type":"ContainerStarted","Data":"9b3494d030ce0c783fa3b4785aa296e57e4f75dcc47ddd9fddf883e73e59ad19"}
Jul 30 19:28:55.753036 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:28:55.752998    2554 generic.go:356] "Generic (PLEG): container finished" podID="e507605f-3466-4993-bbe9-d4a0598f4768" containerID="1cad2ddaecf498e3d25fe55778f6e2e96a4f1b1821d39a4cf14a0d8311c77bd1" exitCode=0
Jul 30 19:28:55.753453 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:28:55.753089    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc" event={"ID":"e507605f-3466-4993-bbe9-d4a0598f4768","Type":"ContainerDied","Data":"1cad2ddaecf498e3d25fe55778f6e2e96a4f1b1821d39a4cf14a0d8311c77bd1"}
Jul 30 19:28:58.766961 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:28:58.766924    2554 generic.go:356] "Generic (PLEG): container finished" podID="e507605f-3466-4993-bbe9-d4a0598f4768" containerID="86ab2a59473d4424ce091314359a72b95f1343e7276fcdbc974690db7b75335c" exitCode=0
Jul 30 19:28:58.767363 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:28:58.767018    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc" event={"ID":"e507605f-3466-4993-bbe9-d4a0598f4768","Type":"ContainerDied","Data":"86ab2a59473d4424ce091314359a72b95f1343e7276fcdbc974690db7b75335c"}
Jul 30 19:29:05.802834 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:05.802799    2554 generic.go:356] "Generic (PLEG): container finished" podID="e507605f-3466-4993-bbe9-d4a0598f4768" containerID="511aeb6e7c74f3a7efb490ce46c7b27f9e9d374ebc872a99f5b5acda4615f591" exitCode=0
Jul 30 19:29:05.803272 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:05.802851    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc" event={"ID":"e507605f-3466-4993-bbe9-d4a0598f4768","Type":"ContainerDied","Data":"511aeb6e7c74f3a7efb490ce46c7b27f9e9d374ebc872a99f5b5acda4615f591"}
Jul 30 19:29:06.932357 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:06.932328    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc"
Jul 30 19:29:07.042258 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:07.042207    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e507605f-3466-4993-bbe9-d4a0598f4768-util\") pod \"e507605f-3466-4993-bbe9-d4a0598f4768\" (UID: \"e507605f-3466-4993-bbe9-d4a0598f4768\") "
Jul 30 19:29:07.042258 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:07.042269    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e507605f-3466-4993-bbe9-d4a0598f4768-bundle\") pod \"e507605f-3466-4993-bbe9-d4a0598f4768\" (UID: \"e507605f-3466-4993-bbe9-d4a0598f4768\") "
Jul 30 19:29:07.042483 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:07.042318    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ktq2p\" (UniqueName: \"kubernetes.io/projected/e507605f-3466-4993-bbe9-d4a0598f4768-kube-api-access-ktq2p\") pod \"e507605f-3466-4993-bbe9-d4a0598f4768\" (UID: \"e507605f-3466-4993-bbe9-d4a0598f4768\") "
Jul 30 19:29:07.043141 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:07.043104    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e507605f-3466-4993-bbe9-d4a0598f4768-bundle" (OuterVolumeSpecName: "bundle") pod "e507605f-3466-4993-bbe9-d4a0598f4768" (UID: "e507605f-3466-4993-bbe9-d4a0598f4768"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:29:07.044598 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:07.044566    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e507605f-3466-4993-bbe9-d4a0598f4768-kube-api-access-ktq2p" (OuterVolumeSpecName: "kube-api-access-ktq2p") pod "e507605f-3466-4993-bbe9-d4a0598f4768" (UID: "e507605f-3466-4993-bbe9-d4a0598f4768"). InnerVolumeSpecName "kube-api-access-ktq2p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:29:07.046715 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:07.046693    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e507605f-3466-4993-bbe9-d4a0598f4768-util" (OuterVolumeSpecName: "util") pod "e507605f-3466-4993-bbe9-d4a0598f4768" (UID: "e507605f-3466-4993-bbe9-d4a0598f4768"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:29:07.143747 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:07.143655    2554 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e507605f-3466-4993-bbe9-d4a0598f4768-util\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:29:07.143747 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:07.143688    2554 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e507605f-3466-4993-bbe9-d4a0598f4768-bundle\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:29:07.143747 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:07.143703    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktq2p\" (UniqueName: \"kubernetes.io/projected/e507605f-3466-4993-bbe9-d4a0598f4768-kube-api-access-ktq2p\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:29:07.812389 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:07.812356    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc" event={"ID":"e507605f-3466-4993-bbe9-d4a0598f4768","Type":"ContainerDied","Data":"9b3494d030ce0c783fa3b4785aa296e57e4f75dcc47ddd9fddf883e73e59ad19"}
Jul 30 19:29:07.812554 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:07.812421    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b3494d030ce0c783fa3b4785aa296e57e4f75dcc47ddd9fddf883e73e59ad19"
Jul 30 19:29:07.812554 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:07.812373    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754098dxc"
Jul 30 19:29:13.791725 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:13.791690    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:14.558624 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:14.558591    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:15.759811 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:15.759776    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4r94b"]
Jul 30 19:29:15.760222 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:15.760091    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e507605f-3466-4993-bbe9-d4a0598f4768" containerName="util"
Jul 30 19:29:15.760222 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:15.760102    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="e507605f-3466-4993-bbe9-d4a0598f4768" containerName="util"
Jul 30 19:29:15.760222 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:15.760113    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e507605f-3466-4993-bbe9-d4a0598f4768" containerName="pull"
Jul 30 19:29:15.760222 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:15.760118    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="e507605f-3466-4993-bbe9-d4a0598f4768" containerName="pull"
Jul 30 19:29:15.760222 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:15.760124    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e507605f-3466-4993-bbe9-d4a0598f4768" containerName="extract"
Jul 30 19:29:15.760222 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:15.760129    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="e507605f-3466-4993-bbe9-d4a0598f4768" containerName="extract"
Jul 30 19:29:15.760222 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:15.760209    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="e507605f-3466-4993-bbe9-d4a0598f4768" containerName="extract"
Jul 30 19:29:15.766817 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:15.766786    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4r94b"
Jul 30 19:29:15.769367 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:15.769340    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 30 19:29:15.769515 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:15.769396    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 30 19:29:15.769515 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:15.769426    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-cljvz\""
Jul 30 19:29:15.777169 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:15.777145    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4r94b"]
Jul 30 19:29:15.822716 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:15.822677    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q25lr\" (UniqueName: \"kubernetes.io/projected/7a9320aa-ff2a-4f39-8896-86cfa625a919-kube-api-access-q25lr\") pod \"cert-manager-operator-controller-manager-55c444bf86-4r94b\" (UID: \"7a9320aa-ff2a-4f39-8896-86cfa625a919\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4r94b"
Jul 30 19:29:15.822902 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:15.822735    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a9320aa-ff2a-4f39-8896-86cfa625a919-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4r94b\" (UID: \"7a9320aa-ff2a-4f39-8896-86cfa625a919\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4r94b"
Jul 30 19:29:15.923562 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:15.923525    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q25lr\" (UniqueName: \"kubernetes.io/projected/7a9320aa-ff2a-4f39-8896-86cfa625a919-kube-api-access-q25lr\") pod \"cert-manager-operator-controller-manager-55c444bf86-4r94b\" (UID: \"7a9320aa-ff2a-4f39-8896-86cfa625a919\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4r94b"
Jul 30 19:29:15.923754 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:15.923582    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a9320aa-ff2a-4f39-8896-86cfa625a919-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4r94b\" (UID: \"7a9320aa-ff2a-4f39-8896-86cfa625a919\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4r94b"
Jul 30 19:29:15.924001 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:15.923983    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a9320aa-ff2a-4f39-8896-86cfa625a919-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4r94b\" (UID: \"7a9320aa-ff2a-4f39-8896-86cfa625a919\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4r94b"
Jul 30 19:29:15.932299 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:15.932273    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q25lr\" (UniqueName: \"kubernetes.io/projected/7a9320aa-ff2a-4f39-8896-86cfa625a919-kube-api-access-q25lr\") pod \"cert-manager-operator-controller-manager-55c444bf86-4r94b\" (UID: \"7a9320aa-ff2a-4f39-8896-86cfa625a919\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4r94b"
Jul 30 19:29:16.077008 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:16.076896    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4r94b"
Jul 30 19:29:16.215298 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:16.215266    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4r94b"]
Jul 30 19:29:16.217890 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:29:16.217855    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a9320aa_ff2a_4f39_8896_86cfa625a919.slice/crio-1e51046d459ec07247a5a95bb044977eb728a6ba425b0b5b0afa2b200ae10f0c WatchSource:0}: Error finding container 1e51046d459ec07247a5a95bb044977eb728a6ba425b0b5b0afa2b200ae10f0c: Status 404 returned error can't find the container with id 1e51046d459ec07247a5a95bb044977eb728a6ba425b0b5b0afa2b200ae10f0c
Jul 30 19:29:16.849529 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:16.849486    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4r94b" event={"ID":"7a9320aa-ff2a-4f39-8896-86cfa625a919","Type":"ContainerStarted","Data":"1e51046d459ec07247a5a95bb044977eb728a6ba425b0b5b0afa2b200ae10f0c"}
Jul 30 19:29:17.408982 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:17.408932    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:17.510003 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:17.509930    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:17.856038 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:17.855999    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4r94b" event={"ID":"7a9320aa-ff2a-4f39-8896-86cfa625a919","Type":"ContainerStarted","Data":"9236543aee8276dda432450e252c639f043744d0b4e9b16a1be3fe05ebe10b2a"}
Jul 30 19:29:18.259806 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:18.259761    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:18.359299 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:18.359260    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:18.464962 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:18.464922    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:19.227707 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.227650    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4r94b" podStartSLOduration=2.705737065 podStartE2EDuration="4.22763018s" totalImagesPullingTime="1.521893115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:29:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:29:17.880579287 +0000 UTC m=+285.845160361" watchObservedRunningTime="2026-07-30 19:29:19.22763018 +0000 UTC m=+287.192211261"
Jul 30 19:29:19.228565 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.228547    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt"]
Jul 30 19:29:19.230925 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.230909    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt"
Jul 30 19:29:19.233251 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.233224    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-m4qtq\""
Jul 30 19:29:19.233373 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.233260    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 30 19:29:19.234136 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.234114    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 30 19:29:19.240348 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.240323    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt"]
Jul 30 19:29:19.354903 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.354867    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23b7d4bb-e7eb-43f2-ace0-d907246ac977-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt\" (UID: \"23b7d4bb-e7eb-43f2-ace0-d907246ac977\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt"
Jul 30 19:29:19.355090 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.354921    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldc6w\" (UniqueName: \"kubernetes.io/projected/23b7d4bb-e7eb-43f2-ace0-d907246ac977-kube-api-access-ldc6w\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt\" (UID: \"23b7d4bb-e7eb-43f2-ace0-d907246ac977\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt"
Jul 30 19:29:19.355090 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.355024    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23b7d4bb-e7eb-43f2-ace0-d907246ac977-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt\" (UID: \"23b7d4bb-e7eb-43f2-ace0-d907246ac977\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt"
Jul 30 19:29:19.456141 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.456091    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23b7d4bb-e7eb-43f2-ace0-d907246ac977-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt\" (UID: \"23b7d4bb-e7eb-43f2-ace0-d907246ac977\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt"
Jul 30 19:29:19.456336 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.456214    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldc6w\" (UniqueName: \"kubernetes.io/projected/23b7d4bb-e7eb-43f2-ace0-d907246ac977-kube-api-access-ldc6w\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt\" (UID: \"23b7d4bb-e7eb-43f2-ace0-d907246ac977\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt"
Jul 30 19:29:19.456336 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.456264    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23b7d4bb-e7eb-43f2-ace0-d907246ac977-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt\" (UID: \"23b7d4bb-e7eb-43f2-ace0-d907246ac977\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt"
Jul 30 19:29:19.456556 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.456532    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23b7d4bb-e7eb-43f2-ace0-d907246ac977-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt\" (UID: \"23b7d4bb-e7eb-43f2-ace0-d907246ac977\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt"
Jul 30 19:29:19.456665 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.456641    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23b7d4bb-e7eb-43f2-ace0-d907246ac977-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt\" (UID: \"23b7d4bb-e7eb-43f2-ace0-d907246ac977\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt"
Jul 30 19:29:19.466215 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.466187    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldc6w\" (UniqueName: \"kubernetes.io/projected/23b7d4bb-e7eb-43f2-ace0-d907246ac977-kube-api-access-ldc6w\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt\" (UID: \"23b7d4bb-e7eb-43f2-ace0-d907246ac977\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt"
Jul 30 19:29:19.474038 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.474016    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-jg8fb"]
Jul 30 19:29:19.476313 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.476298    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-jg8fb"
Jul 30 19:29:19.479515 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.479465    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 30 19:29:19.479515 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.479501    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 30 19:29:19.479662 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.479519    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-xrxrf\""
Jul 30 19:29:19.490823 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.490800    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-jg8fb"]
Jul 30 19:29:19.541080 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.541037    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt"
Jul 30 19:29:19.557268 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.557231    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8cbbf39f-f59e-4766-90ae-2d1b4b1caea8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-jg8fb\" (UID: \"8cbbf39f-f59e-4766-90ae-2d1b4b1caea8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jg8fb"
Jul 30 19:29:19.557422 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.557277    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8xkr\" (UniqueName: \"kubernetes.io/projected/8cbbf39f-f59e-4766-90ae-2d1b4b1caea8-kube-api-access-w8xkr\") pod \"cert-manager-webhook-6486ffd997-jg8fb\" (UID: \"8cbbf39f-f59e-4766-90ae-2d1b4b1caea8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jg8fb"
Jul 30 19:29:19.658702 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.658667    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8cbbf39f-f59e-4766-90ae-2d1b4b1caea8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-jg8fb\" (UID: \"8cbbf39f-f59e-4766-90ae-2d1b4b1caea8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jg8fb"
Jul 30 19:29:19.658887 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.658729    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8xkr\" (UniqueName: \"kubernetes.io/projected/8cbbf39f-f59e-4766-90ae-2d1b4b1caea8-kube-api-access-w8xkr\") pod \"cert-manager-webhook-6486ffd997-jg8fb\" (UID: \"8cbbf39f-f59e-4766-90ae-2d1b4b1caea8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jg8fb"
Jul 30 19:29:19.667623 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.667584    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8cbbf39f-f59e-4766-90ae-2d1b4b1caea8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-jg8fb\" (UID: \"8cbbf39f-f59e-4766-90ae-2d1b4b1caea8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jg8fb"
Jul 30 19:29:19.668038 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.668019    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8xkr\" (UniqueName: \"kubernetes.io/projected/8cbbf39f-f59e-4766-90ae-2d1b4b1caea8-kube-api-access-w8xkr\") pod \"cert-manager-webhook-6486ffd997-jg8fb\" (UID: \"8cbbf39f-f59e-4766-90ae-2d1b4b1caea8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jg8fb"
Jul 30 19:29:19.676094 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.676023    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt"]
Jul 30 19:29:19.678356 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:29:19.678330    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23b7d4bb_e7eb_43f2_ace0_d907246ac977.slice/crio-098f9bfd12f8583a8fc7da710e8284e09bd8a863117b8786be6cb4502cd57424 WatchSource:0}: Error finding container 098f9bfd12f8583a8fc7da710e8284e09bd8a863117b8786be6cb4502cd57424: Status 404 returned error can't find the container with id 098f9bfd12f8583a8fc7da710e8284e09bd8a863117b8786be6cb4502cd57424
Jul 30 19:29:19.797004 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.796966    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-jg8fb"
Jul 30 19:29:19.866887 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.866848    2554 generic.go:356] "Generic (PLEG): container finished" podID="23b7d4bb-e7eb-43f2-ace0-d907246ac977" containerID="9cf77ff6cf09b7566b9470c21a4aa2562e3b64ffa1c8c90514889a4cf5f277a4" exitCode=0
Jul 30 19:29:19.867092 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.866914    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt" event={"ID":"23b7d4bb-e7eb-43f2-ace0-d907246ac977","Type":"ContainerDied","Data":"9cf77ff6cf09b7566b9470c21a4aa2562e3b64ffa1c8c90514889a4cf5f277a4"}
Jul 30 19:29:19.867092 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.867004    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt" event={"ID":"23b7d4bb-e7eb-43f2-ace0-d907246ac977","Type":"ContainerStarted","Data":"098f9bfd12f8583a8fc7da710e8284e09bd8a863117b8786be6cb4502cd57424"}
Jul 30 19:29:19.928464 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:19.928439    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-jg8fb"]
Jul 30 19:29:19.930875 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:29:19.930848    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8cbbf39f_f59e_4766_90ae_2d1b4b1caea8.slice/crio-38a909ab1e2badd5d0fae2d2c6cac532f74830a386cc7a0709789410cb7ccb79 WatchSource:0}: Error finding container 38a909ab1e2badd5d0fae2d2c6cac532f74830a386cc7a0709789410cb7ccb79: Status 404 returned error can't find the container with id 38a909ab1e2badd5d0fae2d2c6cac532f74830a386cc7a0709789410cb7ccb79
Jul 30 19:29:20.706940 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:20.706906    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:20.877908 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:20.877867    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-jg8fb" event={"ID":"8cbbf39f-f59e-4766-90ae-2d1b4b1caea8","Type":"ContainerStarted","Data":"38a909ab1e2badd5d0fae2d2c6cac532f74830a386cc7a0709789410cb7ccb79"}
Jul 30 19:29:20.924479 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:20.924447    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:21.117104 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:21.117063    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hcrvv"]
Jul 30 19:29:21.119657 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:21.119636    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hcrvv"
Jul 30 19:29:21.123221 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:21.123191    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-fwfxd\""
Jul 30 19:29:21.137523 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:21.137502    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hcrvv"]
Jul 30 19:29:21.276011 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:21.275970    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7zhw\" (UniqueName: \"kubernetes.io/projected/fef584a5-c90f-4846-8384-9aefd7e12606-kube-api-access-g7zhw\") pod \"cert-manager-cainjector-559cdd8c87-hcrvv\" (UID: \"fef584a5-c90f-4846-8384-9aefd7e12606\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hcrvv"
Jul 30 19:29:21.276204 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:21.276043    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fef584a5-c90f-4846-8384-9aefd7e12606-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hcrvv\" (UID: \"fef584a5-c90f-4846-8384-9aefd7e12606\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hcrvv"
Jul 30 19:29:21.377598 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:21.377509    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7zhw\" (UniqueName: \"kubernetes.io/projected/fef584a5-c90f-4846-8384-9aefd7e12606-kube-api-access-g7zhw\") pod \"cert-manager-cainjector-559cdd8c87-hcrvv\" (UID: \"fef584a5-c90f-4846-8384-9aefd7e12606\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hcrvv"
Jul 30 19:29:21.377598 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:21.377579    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fef584a5-c90f-4846-8384-9aefd7e12606-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hcrvv\" (UID: \"fef584a5-c90f-4846-8384-9aefd7e12606\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hcrvv"
Jul 30 19:29:21.386534 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:21.386498    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fef584a5-c90f-4846-8384-9aefd7e12606-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hcrvv\" (UID: \"fef584a5-c90f-4846-8384-9aefd7e12606\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hcrvv"
Jul 30 19:29:21.386706 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:21.386612    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7zhw\" (UniqueName: \"kubernetes.io/projected/fef584a5-c90f-4846-8384-9aefd7e12606-kube-api-access-g7zhw\") pod \"cert-manager-cainjector-559cdd8c87-hcrvv\" (UID: \"fef584a5-c90f-4846-8384-9aefd7e12606\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hcrvv"
Jul 30 19:29:21.433245 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:21.433204    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hcrvv"
Jul 30 19:29:23.042268 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:23.042234    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hcrvv"]
Jul 30 19:29:23.044202 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:29:23.044148    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfef584a5_c90f_4846_8384_9aefd7e12606.slice/crio-445734da1677cb01a4ec0bad32bdea47e9024978e82e8eb5e99e1b647d90c33a WatchSource:0}: Error finding container 445734da1677cb01a4ec0bad32bdea47e9024978e82e8eb5e99e1b647d90c33a: Status 404 returned error can't find the container with id 445734da1677cb01a4ec0bad32bdea47e9024978e82e8eb5e99e1b647d90c33a
Jul 30 19:29:23.896267 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:23.896232    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-jg8fb" event={"ID":"8cbbf39f-f59e-4766-90ae-2d1b4b1caea8","Type":"ContainerStarted","Data":"0f9b46effdef8706c18919bdc59401cb0a0c2c1d954f3ce31a5df9599db5857e"}
Jul 30 19:29:23.896472 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:23.896310    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-jg8fb"
Jul 30 19:29:23.898025 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:23.897986    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hcrvv" event={"ID":"fef584a5-c90f-4846-8384-9aefd7e12606","Type":"ContainerStarted","Data":"ce20b9147e610664d8b1b4b51498391f9f4ab7189b79d2baff331a25745b42e3"}
Jul 30 19:29:23.898180 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:23.898030    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hcrvv" event={"ID":"fef584a5-c90f-4846-8384-9aefd7e12606","Type":"ContainerStarted","Data":"445734da1677cb01a4ec0bad32bdea47e9024978e82e8eb5e99e1b647d90c33a"}
Jul 30 19:29:23.899923 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:23.899896    2554 generic.go:356] "Generic (PLEG): container finished" podID="23b7d4bb-e7eb-43f2-ace0-d907246ac977" containerID="ad79c3597cf71978b3bbde1ba79043c67858ebda43ec88acadeecaaf15e5bca8" exitCode=0
Jul 30 19:29:23.900068 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:23.899971    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt" event={"ID":"23b7d4bb-e7eb-43f2-ace0-d907246ac977","Type":"ContainerDied","Data":"ad79c3597cf71978b3bbde1ba79043c67858ebda43ec88acadeecaaf15e5bca8"}
Jul 30 19:29:23.914498 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:23.914451    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-jg8fb" podStartSLOduration=1.942319852 podStartE2EDuration="4.914436854s" totalImagesPullingTime="2.972117002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:29:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:29:23.913134636 +0000 UTC m=+291.877715718" watchObservedRunningTime="2026-07-30 19:29:23.914436854 +0000 UTC m=+291.879017935"
Jul 30 19:29:23.929626 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:23.929577    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hcrvv" podStartSLOduration=2.929558469 podStartE2EDuration="2.929558469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:29:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:29:23.928398318 +0000 UTC m=+291.892979400" watchObservedRunningTime="2026-07-30 19:29:23.929558469 +0000 UTC m=+291.894139550"
Jul 30 19:29:24.199334 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:24.199246    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:24.252023 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:24.251986    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:24.906055 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:24.906013    2554 generic.go:356] "Generic (PLEG): container finished" podID="23b7d4bb-e7eb-43f2-ace0-d907246ac977" containerID="17fc5ef663f817abc1a362f1b6ab37576224b49437d2d989b3b7b806d2b09fb9" exitCode=0
Jul 30 19:29:24.906205 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:24.906079    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt" event={"ID":"23b7d4bb-e7eb-43f2-ace0-d907246ac977","Type":"ContainerDied","Data":"17fc5ef663f817abc1a362f1b6ab37576224b49437d2d989b3b7b806d2b09fb9"}
Jul 30 19:29:26.059568 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:26.059528    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt"
Jul 30 19:29:26.223222 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:26.223190    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldc6w\" (UniqueName: \"kubernetes.io/projected/23b7d4bb-e7eb-43f2-ace0-d907246ac977-kube-api-access-ldc6w\") pod \"23b7d4bb-e7eb-43f2-ace0-d907246ac977\" (UID: \"23b7d4bb-e7eb-43f2-ace0-d907246ac977\") "
Jul 30 19:29:26.223414 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:26.223256    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23b7d4bb-e7eb-43f2-ace0-d907246ac977-bundle\") pod \"23b7d4bb-e7eb-43f2-ace0-d907246ac977\" (UID: \"23b7d4bb-e7eb-43f2-ace0-d907246ac977\") "
Jul 30 19:29:26.223414 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:26.223280    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23b7d4bb-e7eb-43f2-ace0-d907246ac977-util\") pod \"23b7d4bb-e7eb-43f2-ace0-d907246ac977\" (UID: \"23b7d4bb-e7eb-43f2-ace0-d907246ac977\") "
Jul 30 19:29:26.223670 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:26.223645    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23b7d4bb-e7eb-43f2-ace0-d907246ac977-bundle" (OuterVolumeSpecName: "bundle") pod "23b7d4bb-e7eb-43f2-ace0-d907246ac977" (UID: "23b7d4bb-e7eb-43f2-ace0-d907246ac977"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:29:26.225497 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:26.225476    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23b7d4bb-e7eb-43f2-ace0-d907246ac977-kube-api-access-ldc6w" (OuterVolumeSpecName: "kube-api-access-ldc6w") pod "23b7d4bb-e7eb-43f2-ace0-d907246ac977" (UID: "23b7d4bb-e7eb-43f2-ace0-d907246ac977"). InnerVolumeSpecName "kube-api-access-ldc6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:29:26.227923 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:26.227888    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23b7d4bb-e7eb-43f2-ace0-d907246ac977-util" (OuterVolumeSpecName: "util") pod "23b7d4bb-e7eb-43f2-ace0-d907246ac977" (UID: "23b7d4bb-e7eb-43f2-ace0-d907246ac977"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:29:26.324304 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:26.324259    2554 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23b7d4bb-e7eb-43f2-ace0-d907246ac977-bundle\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:29:26.324304 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:26.324296    2554 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23b7d4bb-e7eb-43f2-ace0-d907246ac977-util\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:29:26.324304 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:26.324306    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldc6w\" (UniqueName: \"kubernetes.io/projected/23b7d4bb-e7eb-43f2-ace0-d907246ac977-kube-api-access-ldc6w\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:29:26.917535 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:26.917460    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt"
Jul 30 19:29:26.917535 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:26.917468    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdtqvt" event={"ID":"23b7d4bb-e7eb-43f2-ace0-d907246ac977","Type":"ContainerDied","Data":"098f9bfd12f8583a8fc7da710e8284e09bd8a863117b8786be6cb4502cd57424"}
Jul 30 19:29:26.917535 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:26.917512    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="098f9bfd12f8583a8fc7da710e8284e09bd8a863117b8786be6cb4502cd57424"
Jul 30 19:29:28.313337 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:28.313309    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:29.908828 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:29.908797    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-jg8fb"
Jul 30 19:29:31.483080 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:31.483048    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:31.581856 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:31.581824    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:32.375761 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:32.375732    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/1.log"
Jul 30 19:29:32.376083 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:32.376064    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/1.log"
Jul 30 19:29:32.380735 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:32.380714    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 19:29:32.380884 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:32.380749    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 19:29:32.386430 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:32.386409    2554 kubelet.go:1648] "Image garbage collection succeeded"
Jul 30 19:29:38.382349 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:38.382310    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:40.202866 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.202827    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv"]
Jul 30 19:29:40.203300 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.203155    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23b7d4bb-e7eb-43f2-ace0-d907246ac977" containerName="pull"
Jul 30 19:29:40.203300 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.203167    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="23b7d4bb-e7eb-43f2-ace0-d907246ac977" containerName="pull"
Jul 30 19:29:40.203300 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.203179    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23b7d4bb-e7eb-43f2-ace0-d907246ac977" containerName="extract"
Jul 30 19:29:40.203300 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.203184    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="23b7d4bb-e7eb-43f2-ace0-d907246ac977" containerName="extract"
Jul 30 19:29:40.203300 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.203201    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23b7d4bb-e7eb-43f2-ace0-d907246ac977" containerName="util"
Jul 30 19:29:40.203300 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.203206    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="23b7d4bb-e7eb-43f2-ace0-d907246ac977" containerName="util"
Jul 30 19:29:40.203300 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.203287    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="23b7d4bb-e7eb-43f2-ace0-d907246ac977" containerName="extract"
Jul 30 19:29:40.221599 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.221568    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv"]
Jul 30 19:29:40.221747 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.221705    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv"
Jul 30 19:29:40.224598 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.224575    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 30 19:29:40.224598 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.224576    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 30 19:29:40.225862 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.225843    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-m4qtq\""
Jul 30 19:29:40.242563 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.242534    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9da50adb-8369-4b6d-97f9-fa1af5d1dca3-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv\" (UID: \"9da50adb-8369-4b6d-97f9-fa1af5d1dca3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv"
Jul 30 19:29:40.242720 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.242567    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9da50adb-8369-4b6d-97f9-fa1af5d1dca3-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv\" (UID: \"9da50adb-8369-4b6d-97f9-fa1af5d1dca3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv"
Jul 30 19:29:40.242720 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.242598    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zspgp\" (UniqueName: \"kubernetes.io/projected/9da50adb-8369-4b6d-97f9-fa1af5d1dca3-kube-api-access-zspgp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv\" (UID: \"9da50adb-8369-4b6d-97f9-fa1af5d1dca3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv"
Jul 30 19:29:40.343696 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.343658    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9da50adb-8369-4b6d-97f9-fa1af5d1dca3-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv\" (UID: \"9da50adb-8369-4b6d-97f9-fa1af5d1dca3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv"
Jul 30 19:29:40.343696 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.343695    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9da50adb-8369-4b6d-97f9-fa1af5d1dca3-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv\" (UID: \"9da50adb-8369-4b6d-97f9-fa1af5d1dca3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv"
Jul 30 19:29:40.343977 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.343716    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zspgp\" (UniqueName: \"kubernetes.io/projected/9da50adb-8369-4b6d-97f9-fa1af5d1dca3-kube-api-access-zspgp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv\" (UID: \"9da50adb-8369-4b6d-97f9-fa1af5d1dca3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv"
Jul 30 19:29:40.344092 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.344070    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9da50adb-8369-4b6d-97f9-fa1af5d1dca3-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv\" (UID: \"9da50adb-8369-4b6d-97f9-fa1af5d1dca3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv"
Jul 30 19:29:40.344175 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.344151    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9da50adb-8369-4b6d-97f9-fa1af5d1dca3-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv\" (UID: \"9da50adb-8369-4b6d-97f9-fa1af5d1dca3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv"
Jul 30 19:29:40.351906 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.351872    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zspgp\" (UniqueName: \"kubernetes.io/projected/9da50adb-8369-4b6d-97f9-fa1af5d1dca3-kube-api-access-zspgp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv\" (UID: \"9da50adb-8369-4b6d-97f9-fa1af5d1dca3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv"
Jul 30 19:29:40.364561 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.364528    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:40.531741 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.531705    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv"
Jul 30 19:29:40.687976 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.687925    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv"]
Jul 30 19:29:40.689354 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:29:40.689330    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9da50adb_8369_4b6d_97f9_fa1af5d1dca3.slice/crio-69e9ec54080be7f259c32e7775d4c4dba3f9de800dd735427c15925d4d98d17d WatchSource:0}: Error finding container 69e9ec54080be7f259c32e7775d4c4dba3f9de800dd735427c15925d4d98d17d: Status 404 returned error can't find the container with id 69e9ec54080be7f259c32e7775d4c4dba3f9de800dd735427c15925d4d98d17d
Jul 30 19:29:40.981845 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.981811    2554 generic.go:356] "Generic (PLEG): container finished" podID="9da50adb-8369-4b6d-97f9-fa1af5d1dca3" containerID="7411c62f0a39a28e89670ee0e7a4117fdf906ec4484c681209bc8e2cd6db7d07" exitCode=0
Jul 30 19:29:40.982061 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.981889    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv" event={"ID":"9da50adb-8369-4b6d-97f9-fa1af5d1dca3","Type":"ContainerDied","Data":"7411c62f0a39a28e89670ee0e7a4117fdf906ec4484c681209bc8e2cd6db7d07"}
Jul 30 19:29:40.982061 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.981985    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv" event={"ID":"9da50adb-8369-4b6d-97f9-fa1af5d1dca3","Type":"ContainerStarted","Data":"69e9ec54080be7f259c32e7775d4c4dba3f9de800dd735427c15925d4d98d17d"}
Jul 30 19:29:40.982432 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:40.982415    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 19:29:41.988320 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:41.988286    2554 generic.go:356] "Generic (PLEG): container finished" podID="9da50adb-8369-4b6d-97f9-fa1af5d1dca3" containerID="c83648bf7dcdb0dd8baf869fe8ba39cd0deaa7654d3532fc74564d707e556f40" exitCode=0
Jul 30 19:29:41.988703 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:41.988374    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv" event={"ID":"9da50adb-8369-4b6d-97f9-fa1af5d1dca3","Type":"ContainerDied","Data":"c83648bf7dcdb0dd8baf869fe8ba39cd0deaa7654d3532fc74564d707e556f40"}
Jul 30 19:29:42.994529 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:42.994496    2554 generic.go:356] "Generic (PLEG): container finished" podID="9da50adb-8369-4b6d-97f9-fa1af5d1dca3" containerID="0b97c743e370cc11d489c2946a5ff3583de77f1886a0437d3dbdcd7d38eadabc" exitCode=0
Jul 30 19:29:42.995030 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:42.994584    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv" event={"ID":"9da50adb-8369-4b6d-97f9-fa1af5d1dca3","Type":"ContainerDied","Data":"0b97c743e370cc11d489c2946a5ff3583de77f1886a0437d3dbdcd7d38eadabc"}
Jul 30 19:29:44.129421 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:44.129396    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv"
Jul 30 19:29:44.178104 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:44.178066    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9da50adb-8369-4b6d-97f9-fa1af5d1dca3-util\") pod \"9da50adb-8369-4b6d-97f9-fa1af5d1dca3\" (UID: \"9da50adb-8369-4b6d-97f9-fa1af5d1dca3\") "
Jul 30 19:29:44.178286 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:44.178115    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9da50adb-8369-4b6d-97f9-fa1af5d1dca3-bundle\") pod \"9da50adb-8369-4b6d-97f9-fa1af5d1dca3\" (UID: \"9da50adb-8369-4b6d-97f9-fa1af5d1dca3\") "
Jul 30 19:29:44.178286 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:44.178141    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zspgp\" (UniqueName: \"kubernetes.io/projected/9da50adb-8369-4b6d-97f9-fa1af5d1dca3-kube-api-access-zspgp\") pod \"9da50adb-8369-4b6d-97f9-fa1af5d1dca3\" (UID: \"9da50adb-8369-4b6d-97f9-fa1af5d1dca3\") "
Jul 30 19:29:44.178989 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:44.178937    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9da50adb-8369-4b6d-97f9-fa1af5d1dca3-bundle" (OuterVolumeSpecName: "bundle") pod "9da50adb-8369-4b6d-97f9-fa1af5d1dca3" (UID: "9da50adb-8369-4b6d-97f9-fa1af5d1dca3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:29:44.180260 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:44.180236    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9da50adb-8369-4b6d-97f9-fa1af5d1dca3-kube-api-access-zspgp" (OuterVolumeSpecName: "kube-api-access-zspgp") pod "9da50adb-8369-4b6d-97f9-fa1af5d1dca3" (UID: "9da50adb-8369-4b6d-97f9-fa1af5d1dca3"). InnerVolumeSpecName "kube-api-access-zspgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:29:44.183847 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:44.183811    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9da50adb-8369-4b6d-97f9-fa1af5d1dca3-util" (OuterVolumeSpecName: "util") pod "9da50adb-8369-4b6d-97f9-fa1af5d1dca3" (UID: "9da50adb-8369-4b6d-97f9-fa1af5d1dca3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:29:44.278856 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:44.278778    2554 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9da50adb-8369-4b6d-97f9-fa1af5d1dca3-util\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:29:44.278856 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:44.278808    2554 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9da50adb-8369-4b6d-97f9-fa1af5d1dca3-bundle\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:29:44.278856 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:44.278818    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zspgp\" (UniqueName: \"kubernetes.io/projected/9da50adb-8369-4b6d-97f9-fa1af5d1dca3-kube-api-access-zspgp\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:29:45.005385 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:45.005351    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv"
Jul 30 19:29:45.005385 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:45.005364    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbtrbv" event={"ID":"9da50adb-8369-4b6d-97f9-fa1af5d1dca3","Type":"ContainerDied","Data":"69e9ec54080be7f259c32e7775d4c4dba3f9de800dd735427c15925d4d98d17d"}
Jul 30 19:29:45.005617 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:45.005411    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="69e9ec54080be7f259c32e7775d4c4dba3f9de800dd735427c15925d4d98d17d"
Jul 30 19:29:46.519093 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:46.519047    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:46.574778 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:46.574747    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:54.036478 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.036432    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:54.134703 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.134667    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:54.561451 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.561418    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:54.656551 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.656511    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:54.661236 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.661211    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-659fd749fb-tnqm6"]
Jul 30 19:29:54.661562 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.661549    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9da50adb-8369-4b6d-97f9-fa1af5d1dca3" containerName="util"
Jul 30 19:29:54.661615 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.661564    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="9da50adb-8369-4b6d-97f9-fa1af5d1dca3" containerName="util"
Jul 30 19:29:54.661615 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.661582    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9da50adb-8369-4b6d-97f9-fa1af5d1dca3" containerName="pull"
Jul 30 19:29:54.661615 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.661588    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="9da50adb-8369-4b6d-97f9-fa1af5d1dca3" containerName="pull"
Jul 30 19:29:54.661615 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.661601    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9da50adb-8369-4b6d-97f9-fa1af5d1dca3" containerName="extract"
Jul 30 19:29:54.661615 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.661606    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="9da50adb-8369-4b6d-97f9-fa1af5d1dca3" containerName="extract"
Jul 30 19:29:54.661810 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.661654    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="9da50adb-8369-4b6d-97f9-fa1af5d1dca3" containerName="extract"
Jul 30 19:29:54.664478 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.664461    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-659fd749fb-tnqm6"
Jul 30 19:29:54.669230 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.669209    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 30 19:29:54.669529 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.669514    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 30 19:29:54.670372 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.670341    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 30 19:29:54.670473 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.670379    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 30 19:29:54.670473 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.670348    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-4kpj4\""
Jul 30 19:29:54.670473 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.670408    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 30 19:29:54.679228 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.679202    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-659fd749fb-tnqm6"]
Jul 30 19:29:54.735284 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.735253    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:54.766384 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.766341    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8e322bc6-457c-418a-abd4-63acc15d5cca-manager-config\") pod \"lws-controller-manager-659fd749fb-tnqm6\" (UID: \"8e322bc6-457c-418a-abd4-63acc15d5cca\") " pod="openshift-lws-operator/lws-controller-manager-659fd749fb-tnqm6"
Jul 30 19:29:54.766556 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.766470    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e322bc6-457c-418a-abd4-63acc15d5cca-cert\") pod \"lws-controller-manager-659fd749fb-tnqm6\" (UID: \"8e322bc6-457c-418a-abd4-63acc15d5cca\") " pod="openshift-lws-operator/lws-controller-manager-659fd749fb-tnqm6"
Jul 30 19:29:54.766556 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.766510    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8e322bc6-457c-418a-abd4-63acc15d5cca-metrics-cert\") pod \"lws-controller-manager-659fd749fb-tnqm6\" (UID: \"8e322bc6-457c-418a-abd4-63acc15d5cca\") " pod="openshift-lws-operator/lws-controller-manager-659fd749fb-tnqm6"
Jul 30 19:29:54.766556 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.766530    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9zkd\" (UniqueName: \"kubernetes.io/projected/8e322bc6-457c-418a-abd4-63acc15d5cca-kube-api-access-l9zkd\") pod \"lws-controller-manager-659fd749fb-tnqm6\" (UID: \"8e322bc6-457c-418a-abd4-63acc15d5cca\") " pod="openshift-lws-operator/lws-controller-manager-659fd749fb-tnqm6"
Jul 30 19:29:54.867985 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.867870    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e322bc6-457c-418a-abd4-63acc15d5cca-cert\") pod \"lws-controller-manager-659fd749fb-tnqm6\" (UID: \"8e322bc6-457c-418a-abd4-63acc15d5cca\") " pod="openshift-lws-operator/lws-controller-manager-659fd749fb-tnqm6"
Jul 30 19:29:54.867985 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.867929    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8e322bc6-457c-418a-abd4-63acc15d5cca-metrics-cert\") pod \"lws-controller-manager-659fd749fb-tnqm6\" (UID: \"8e322bc6-457c-418a-abd4-63acc15d5cca\") " pod="openshift-lws-operator/lws-controller-manager-659fd749fb-tnqm6"
Jul 30 19:29:54.867985 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.867972    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9zkd\" (UniqueName: \"kubernetes.io/projected/8e322bc6-457c-418a-abd4-63acc15d5cca-kube-api-access-l9zkd\") pod \"lws-controller-manager-659fd749fb-tnqm6\" (UID: \"8e322bc6-457c-418a-abd4-63acc15d5cca\") " pod="openshift-lws-operator/lws-controller-manager-659fd749fb-tnqm6"
Jul 30 19:29:54.868199 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.868011    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8e322bc6-457c-418a-abd4-63acc15d5cca-manager-config\") pod \"lws-controller-manager-659fd749fb-tnqm6\" (UID: \"8e322bc6-457c-418a-abd4-63acc15d5cca\") " pod="openshift-lws-operator/lws-controller-manager-659fd749fb-tnqm6"
Jul 30 19:29:54.868806 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.868780    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8e322bc6-457c-418a-abd4-63acc15d5cca-manager-config\") pod \"lws-controller-manager-659fd749fb-tnqm6\" (UID: \"8e322bc6-457c-418a-abd4-63acc15d5cca\") " pod="openshift-lws-operator/lws-controller-manager-659fd749fb-tnqm6"
Jul 30 19:29:54.870457 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.870436    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e322bc6-457c-418a-abd4-63acc15d5cca-cert\") pod \"lws-controller-manager-659fd749fb-tnqm6\" (UID: \"8e322bc6-457c-418a-abd4-63acc15d5cca\") " pod="openshift-lws-operator/lws-controller-manager-659fd749fb-tnqm6"
Jul 30 19:29:54.870542 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.870480    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8e322bc6-457c-418a-abd4-63acc15d5cca-metrics-cert\") pod \"lws-controller-manager-659fd749fb-tnqm6\" (UID: \"8e322bc6-457c-418a-abd4-63acc15d5cca\") " pod="openshift-lws-operator/lws-controller-manager-659fd749fb-tnqm6"
Jul 30 19:29:54.878249 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.878220    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9zkd\" (UniqueName: \"kubernetes.io/projected/8e322bc6-457c-418a-abd4-63acc15d5cca-kube-api-access-l9zkd\") pod \"lws-controller-manager-659fd749fb-tnqm6\" (UID: \"8e322bc6-457c-418a-abd4-63acc15d5cca\") " pod="openshift-lws-operator/lws-controller-manager-659fd749fb-tnqm6"
Jul 30 19:29:54.931439 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.931400    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:54.974238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:54.974199    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-659fd749fb-tnqm6"
Jul 30 19:29:55.037304 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.037271    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:55.097518 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.097485    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr"]
Jul 30 19:29:55.101253 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.101231    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr"
Jul 30 19:29:55.104373 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.104347    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 30 19:29:55.104714 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.104694    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-29w77\""
Jul 30 19:29:55.104805 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.104714    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 30 19:29:55.106488 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.106469    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 30 19:29:55.112329 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.112311    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 30 19:29:55.138622 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.138562    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr"]
Jul 30 19:29:55.139584 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:29:55.139555    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e322bc6_457c_418a_abd4_63acc15d5cca.slice/crio-33ebb26bf8dbc1e08882ef6e5886139397e5d5e5c468b582a6e97266e407e316 WatchSource:0}: Error finding container 33ebb26bf8dbc1e08882ef6e5886139397e5d5e5c468b582a6e97266e407e316: Status 404 returned error can't find the container with id 33ebb26bf8dbc1e08882ef6e5886139397e5d5e5c468b582a6e97266e407e316
Jul 30 19:29:55.140737 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.140714    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-659fd749fb-tnqm6"]
Jul 30 19:29:55.171654 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.171617    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a1391f7f-8cee-46c8-940f-167d78cd70e3-manifests\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr\" (UID: \"a1391f7f-8cee-46c8-940f-167d78cd70e3\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr"
Jul 30 19:29:55.171819 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.171711    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1391f7f-8cee-46c8-940f-167d78cd70e3-tmp\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr\" (UID: \"a1391f7f-8cee-46c8-940f-167d78cd70e3\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr"
Jul 30 19:29:55.171819 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.171758    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a1391f7f-8cee-46c8-940f-167d78cd70e3-webhook-cert\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr\" (UID: \"a1391f7f-8cee-46c8-940f-167d78cd70e3\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr"
Jul 30 19:29:55.171896 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.171831    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a1391f7f-8cee-46c8-940f-167d78cd70e3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr\" (UID: \"a1391f7f-8cee-46c8-940f-167d78cd70e3\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr"
Jul 30 19:29:55.171896 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.171872    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lk6np\" (UniqueName: \"kubernetes.io/projected/a1391f7f-8cee-46c8-940f-167d78cd70e3-kube-api-access-lk6np\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr\" (UID: \"a1391f7f-8cee-46c8-940f-167d78cd70e3\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr"
Jul 30 19:29:55.272896 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.272862    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a1391f7f-8cee-46c8-940f-167d78cd70e3-manifests\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr\" (UID: \"a1391f7f-8cee-46c8-940f-167d78cd70e3\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr"
Jul 30 19:29:55.272896 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.272906    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1391f7f-8cee-46c8-940f-167d78cd70e3-tmp\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr\" (UID: \"a1391f7f-8cee-46c8-940f-167d78cd70e3\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr"
Jul 30 19:29:55.273158 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.272928    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a1391f7f-8cee-46c8-940f-167d78cd70e3-webhook-cert\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr\" (UID: \"a1391f7f-8cee-46c8-940f-167d78cd70e3\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr"
Jul 30 19:29:55.273158 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.272987    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a1391f7f-8cee-46c8-940f-167d78cd70e3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr\" (UID: \"a1391f7f-8cee-46c8-940f-167d78cd70e3\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr"
Jul 30 19:29:55.273158 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.273014    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lk6np\" (UniqueName: \"kubernetes.io/projected/a1391f7f-8cee-46c8-940f-167d78cd70e3-kube-api-access-lk6np\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr\" (UID: \"a1391f7f-8cee-46c8-940f-167d78cd70e3\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr"
Jul 30 19:29:55.273304 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.273282    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a1391f7f-8cee-46c8-940f-167d78cd70e3-manifests\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr\" (UID: \"a1391f7f-8cee-46c8-940f-167d78cd70e3\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr"
Jul 30 19:29:55.275238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.275209    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1391f7f-8cee-46c8-940f-167d78cd70e3-tmp\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr\" (UID: \"a1391f7f-8cee-46c8-940f-167d78cd70e3\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr"
Jul 30 19:29:55.275603 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.275575    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a1391f7f-8cee-46c8-940f-167d78cd70e3-webhook-cert\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr\" (UID: \"a1391f7f-8cee-46c8-940f-167d78cd70e3\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr"
Jul 30 19:29:55.275651 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.275575    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a1391f7f-8cee-46c8-940f-167d78cd70e3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr\" (UID: \"a1391f7f-8cee-46c8-940f-167d78cd70e3\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr"
Jul 30 19:29:55.280818 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.280785    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:55.281252 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.281235    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lk6np\" (UniqueName: \"kubernetes.io/projected/a1391f7f-8cee-46c8-940f-167d78cd70e3-kube-api-access-lk6np\") pod \"opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr\" (UID: \"a1391f7f-8cee-46c8-940f-167d78cd70e3\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr"
Jul 30 19:29:55.389501 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.389416    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:55.412493 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.412444    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr"
Jul 30 19:29:55.481407 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.481371    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:55.560268 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.560228    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:55.561713 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.561691    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr"]
Jul 30 19:29:55.563969 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:29:55.563929    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1391f7f_8cee_46c8_940f_167d78cd70e3.slice/crio-6ed114d5da534cd4e75c24be0bf0a83752973c0489c8f45048b6fa653fb68737 WatchSource:0}: Error finding container 6ed114d5da534cd4e75c24be0bf0a83752973c0489c8f45048b6fa653fb68737: Status 404 returned error can't find the container with id 6ed114d5da534cd4e75c24be0bf0a83752973c0489c8f45048b6fa653fb68737
Jul 30 19:29:55.612083 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.612052    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:55.723274 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.723243    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:55.813573 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:55.813533    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:29:56.063433 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:56.063341    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-659fd749fb-tnqm6" event={"ID":"8e322bc6-457c-418a-abd4-63acc15d5cca","Type":"ContainerStarted","Data":"33ebb26bf8dbc1e08882ef6e5886139397e5d5e5c468b582a6e97266e407e316"}
Jul 30 19:29:56.064939 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:56.064896    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr" event={"ID":"a1391f7f-8cee-46c8-940f-167d78cd70e3","Type":"ContainerStarted","Data":"6ed114d5da534cd4e75c24be0bf0a83752973c0489c8f45048b6fa653fb68737"}
Jul 30 19:29:57.073191 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:57.072985    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-659fd749fb-tnqm6" event={"ID":"8e322bc6-457c-418a-abd4-63acc15d5cca","Type":"ContainerStarted","Data":"438245151a42b36e333a95063789bd23b25563c65142aad9f51371ecbd691f91"}
Jul 30 19:29:57.073666 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:57.073236    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-659fd749fb-tnqm6"
Jul 30 19:29:57.090786 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:57.090726    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-659fd749fb-tnqm6" podStartSLOduration=1.383865446 podStartE2EDuration="3.09070653s" totalImagesPullingTime="1.706841084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:29:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:29:57.088684515 +0000 UTC m=+325.053265596" watchObservedRunningTime="2026-07-30 19:29:57.09070653 +0000 UTC m=+325.055287611"
Jul 30 19:29:59.084907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:29:59.084867    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr" event={"ID":"a1391f7f-8cee-46c8-940f-167d78cd70e3","Type":"ContainerStarted","Data":"c59404c32388e3da74fbfcc165ddbd49051a71dc19bd8c543bfbebe056be5fa7"}
Jul 30 19:30:00.091698 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:00.091594    2554 generic.go:356] "Generic (PLEG): container finished" podID="a1391f7f-8cee-46c8-940f-167d78cd70e3" containerID="c59404c32388e3da74fbfcc165ddbd49051a71dc19bd8c543bfbebe056be5fa7" exitCode=0
Jul 30 19:30:00.091698 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:00.091664    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr" event={"ID":"a1391f7f-8cee-46c8-940f-167d78cd70e3","Type":"ContainerDied","Data":"c59404c32388e3da74fbfcc165ddbd49051a71dc19bd8c543bfbebe056be5fa7"}
Jul 30 19:30:01.098459 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:01.098425    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr" event={"ID":"a1391f7f-8cee-46c8-940f-167d78cd70e3","Type":"ContainerStarted","Data":"aa2a9c133087fdf8587d18cff5b55fc398afd6407eef93e2fd4820ec74d6ef85"}
Jul 30 19:30:01.098864 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:01.098523    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr"
Jul 30 19:30:07.165083 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:07.165032    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:30:07.260254 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:07.260218    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:30:08.080843 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:08.080810    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-659fd749fb-tnqm6"
Jul 30 19:30:08.096869 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:08.096813    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr" podStartSLOduration=8.50715041 podStartE2EDuration="13.096791642s" totalImagesPullingTime="3.470210682s" totalInitContainerRuntime="1.11943055s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:29:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:30:01.120775783 +0000 UTC m=+329.085356863" watchObservedRunningTime="2026-07-30 19:30:08.096791642 +0000 UTC m=+336.061372724"
Jul 30 19:30:12.105895 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:12.105859    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr"
Jul 30 19:30:14.297314 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:14.297282    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:30:14.452876 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:14.452841    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:30:15.084047 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:15.084011    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:30:15.189423 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:15.189389    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:30:15.293262 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:15.293226    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:30:15.734838 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:15.734810    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:30:15.789113 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:15.789081    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:30:15.889534 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:15.889501    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:30:24.903389 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:24.903304    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:30:25.018315 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:25.018277    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:30:25.173316 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:25.173228    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:30:25.473471 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:25.473429    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:30:25.570011 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:25.569945    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:30:25.753929 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:25.753843    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:30:26.250250 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:26.250211    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:30:26.348885 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:26.348854    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:30:26.443059 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:26.443026    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:30:27.278530 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:27.278493    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:30:27.340312 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:27.340273    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:30:28.385339 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:28.385294    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:30:32.673029 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:32.672990    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:30:32.790674 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:32.790643    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:30:32.952068 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:32.951995    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"]
Jul 30 19:30:32.959845 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:32.959819    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:32.962312 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:32.962287    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 19:30:32.962418 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:32.962287    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-fpjdk\""
Jul 30 19:30:32.967795 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:32.967768    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"]
Jul 30 19:30:32.976133 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:32.976103    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:30:33.012074 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.012034    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c97707f-d73f-4d27-84f7-3152b9b709ea-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv\" (UID: \"2c97707f-d73f-4d27-84f7-3152b9b709ea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:33.012275 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.012099    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c97707f-d73f-4d27-84f7-3152b9b709ea-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv\" (UID: \"2c97707f-d73f-4d27-84f7-3152b9b709ea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:33.012275 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.012133    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c97707f-d73f-4d27-84f7-3152b9b709ea-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv\" (UID: \"2c97707f-d73f-4d27-84f7-3152b9b709ea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:33.012275 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.012156    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj9gh\" (UniqueName: \"kubernetes.io/projected/2c97707f-d73f-4d27-84f7-3152b9b709ea-kube-api-access-zj9gh\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv\" (UID: \"2c97707f-d73f-4d27-84f7-3152b9b709ea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:33.012275 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.012234    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c97707f-d73f-4d27-84f7-3152b9b709ea-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv\" (UID: \"2c97707f-d73f-4d27-84f7-3152b9b709ea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:33.012499 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.012324    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c97707f-d73f-4d27-84f7-3152b9b709ea-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv\" (UID: \"2c97707f-d73f-4d27-84f7-3152b9b709ea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:33.012499 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.012387    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c97707f-d73f-4d27-84f7-3152b9b709ea-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv\" (UID: \"2c97707f-d73f-4d27-84f7-3152b9b709ea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:33.012499 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.012425    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c97707f-d73f-4d27-84f7-3152b9b709ea-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv\" (UID: \"2c97707f-d73f-4d27-84f7-3152b9b709ea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:33.012499 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.012449    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c97707f-d73f-4d27-84f7-3152b9b709ea-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv\" (UID: \"2c97707f-d73f-4d27-84f7-3152b9b709ea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:33.113513 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.113470    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c97707f-d73f-4d27-84f7-3152b9b709ea-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv\" (UID: \"2c97707f-d73f-4d27-84f7-3152b9b709ea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:33.113513 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.113511    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c97707f-d73f-4d27-84f7-3152b9b709ea-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv\" (UID: \"2c97707f-d73f-4d27-84f7-3152b9b709ea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:33.113776 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.113532    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c97707f-d73f-4d27-84f7-3152b9b709ea-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv\" (UID: \"2c97707f-d73f-4d27-84f7-3152b9b709ea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:33.113776 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.113578    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c97707f-d73f-4d27-84f7-3152b9b709ea-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv\" (UID: \"2c97707f-d73f-4d27-84f7-3152b9b709ea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:33.113776 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.113605    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c97707f-d73f-4d27-84f7-3152b9b709ea-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv\" (UID: \"2c97707f-d73f-4d27-84f7-3152b9b709ea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:33.113776 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.113634    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj9gh\" (UniqueName: \"kubernetes.io/projected/2c97707f-d73f-4d27-84f7-3152b9b709ea-kube-api-access-zj9gh\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv\" (UID: \"2c97707f-d73f-4d27-84f7-3152b9b709ea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:33.113776 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.113691    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c97707f-d73f-4d27-84f7-3152b9b709ea-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv\" (UID: \"2c97707f-d73f-4d27-84f7-3152b9b709ea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:33.113776 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.113720    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c97707f-d73f-4d27-84f7-3152b9b709ea-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv\" (UID: \"2c97707f-d73f-4d27-84f7-3152b9b709ea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:33.113776 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.113760    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c97707f-d73f-4d27-84f7-3152b9b709ea-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv\" (UID: \"2c97707f-d73f-4d27-84f7-3152b9b709ea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:33.114175 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.114046    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c97707f-d73f-4d27-84f7-3152b9b709ea-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv\" (UID: \"2c97707f-d73f-4d27-84f7-3152b9b709ea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:33.114175 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.114117    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c97707f-d73f-4d27-84f7-3152b9b709ea-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv\" (UID: \"2c97707f-d73f-4d27-84f7-3152b9b709ea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:33.114260 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.114181    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c97707f-d73f-4d27-84f7-3152b9b709ea-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv\" (UID: \"2c97707f-d73f-4d27-84f7-3152b9b709ea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:33.114295 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.114266    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c97707f-d73f-4d27-84f7-3152b9b709ea-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv\" (UID: \"2c97707f-d73f-4d27-84f7-3152b9b709ea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:33.114361 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.114339    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c97707f-d73f-4d27-84f7-3152b9b709ea-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv\" (UID: \"2c97707f-d73f-4d27-84f7-3152b9b709ea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:33.116071 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.116052    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c97707f-d73f-4d27-84f7-3152b9b709ea-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv\" (UID: \"2c97707f-d73f-4d27-84f7-3152b9b709ea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:33.116211 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.116192    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c97707f-d73f-4d27-84f7-3152b9b709ea-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv\" (UID: \"2c97707f-d73f-4d27-84f7-3152b9b709ea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:33.121795 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.121771    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c97707f-d73f-4d27-84f7-3152b9b709ea-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv\" (UID: \"2c97707f-d73f-4d27-84f7-3152b9b709ea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:33.122389 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.122372    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj9gh\" (UniqueName: \"kubernetes.io/projected/2c97707f-d73f-4d27-84f7-3152b9b709ea-kube-api-access-zj9gh\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv\" (UID: \"2c97707f-d73f-4d27-84f7-3152b9b709ea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:33.180187 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.180149    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:30:33.271537 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.271500    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:33.307989 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.307927    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:30:33.414784 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:33.414760    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"]
Jul 30 19:30:33.417117 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:30:33.417083    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c97707f_d73f_4d27_84f7_3152b9b709ea.slice/crio-9a13142b38e86a96027680cdbb6440bda67def5afaa3a5555a38780f772c2d76 WatchSource:0}: Error finding container 9a13142b38e86a96027680cdbb6440bda67def5afaa3a5555a38780f772c2d76: Status 404 returned error can't find the container with id 9a13142b38e86a96027680cdbb6440bda67def5afaa3a5555a38780f772c2d76
Jul 30 19:30:34.255323 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:34.255285    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv" event={"ID":"2c97707f-d73f-4d27-84f7-3152b9b709ea","Type":"ContainerStarted","Data":"9a13142b38e86a96027680cdbb6440bda67def5afaa3a5555a38780f772c2d76"}
Jul 30 19:30:36.381446 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:36.381405    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 19:30:36.381717 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:36.381480    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 19:30:36.381717 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:36.381506    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 19:30:37.271766 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:37.271726    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv" event={"ID":"2c97707f-d73f-4d27-84f7-3152b9b709ea","Type":"ContainerStarted","Data":"5d4133e8cd6146aceea4fb780c3c9c369539e6e98e8e6b1fe801f70ec3144460"}
Jul 30 19:30:38.271839 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:38.271794    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:38.277532 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:38.277508    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:38.298807 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:38.298753    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv" podStartSLOduration=3.336853854 podStartE2EDuration="6.298736007s" totalImagesPullingTime="2.961882153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:30:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:30:37.290782747 +0000 UTC m=+365.255363829" watchObservedRunningTime="2026-07-30 19:30:38.298736007 +0000 UTC m=+366.263317087"
Jul 30 19:30:39.281856 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:39.281825    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:30:39.282785 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:30:39.282767    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv"
Jul 30 19:31:10.922652 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:10.922613    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f8954cf7f-gnzjs"]
Jul 30 19:31:10.933088 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:10.933057    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f8954cf7f-gnzjs"
Jul 30 19:31:10.938888 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:10.938850    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f8954cf7f-gnzjs"]
Jul 30 19:31:10.975352 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:10.975267    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:31:11.027511 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:11.027394    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:31:11.041439 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:11.041407    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10-trusted-ca-bundle\") pod \"console-5f8954cf7f-gnzjs\" (UID: \"8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10\") " pod="openshift-console/console-5f8954cf7f-gnzjs"
Jul 30 19:31:11.041569 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:11.041449    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10-console-serving-cert\") pod \"console-5f8954cf7f-gnzjs\" (UID: \"8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10\") " pod="openshift-console/console-5f8954cf7f-gnzjs"
Jul 30 19:31:11.041569 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:11.041491    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10-console-oauth-config\") pod \"console-5f8954cf7f-gnzjs\" (UID: \"8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10\") " pod="openshift-console/console-5f8954cf7f-gnzjs"
Jul 30 19:31:11.041569 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:11.041528    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10-oauth-serving-cert\") pod \"console-5f8954cf7f-gnzjs\" (UID: \"8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10\") " pod="openshift-console/console-5f8954cf7f-gnzjs"
Jul 30 19:31:11.041569 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:11.041554    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2wx7\" (UniqueName: \"kubernetes.io/projected/8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10-kube-api-access-m2wx7\") pod \"console-5f8954cf7f-gnzjs\" (UID: \"8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10\") " pod="openshift-console/console-5f8954cf7f-gnzjs"
Jul 30 19:31:11.041902 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:11.041641    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10-console-config\") pod \"console-5f8954cf7f-gnzjs\" (UID: \"8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10\") " pod="openshift-console/console-5f8954cf7f-gnzjs"
Jul 30 19:31:11.041902 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:11.041696    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10-service-ca\") pod \"console-5f8954cf7f-gnzjs\" (UID: \"8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10\") " pod="openshift-console/console-5f8954cf7f-gnzjs"
Jul 30 19:31:11.119049 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:11.119007    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:31:11.143204 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:11.143161    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10-service-ca\") pod \"console-5f8954cf7f-gnzjs\" (UID: \"8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10\") " pod="openshift-console/console-5f8954cf7f-gnzjs"
Jul 30 19:31:11.143389 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:11.143223    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10-trusted-ca-bundle\") pod \"console-5f8954cf7f-gnzjs\" (UID: \"8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10\") " pod="openshift-console/console-5f8954cf7f-gnzjs"
Jul 30 19:31:11.143389 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:11.143256    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10-console-serving-cert\") pod \"console-5f8954cf7f-gnzjs\" (UID: \"8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10\") " pod="openshift-console/console-5f8954cf7f-gnzjs"
Jul 30 19:31:11.143389 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:11.143277    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10-console-oauth-config\") pod \"console-5f8954cf7f-gnzjs\" (UID: \"8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10\") " pod="openshift-console/console-5f8954cf7f-gnzjs"
Jul 30 19:31:11.143389 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:11.143324    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10-oauth-serving-cert\") pod \"console-5f8954cf7f-gnzjs\" (UID: \"8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10\") " pod="openshift-console/console-5f8954cf7f-gnzjs"
Jul 30 19:31:11.143389 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:11.143363    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2wx7\" (UniqueName: \"kubernetes.io/projected/8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10-kube-api-access-m2wx7\") pod \"console-5f8954cf7f-gnzjs\" (UID: \"8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10\") " pod="openshift-console/console-5f8954cf7f-gnzjs"
Jul 30 19:31:11.143700 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:11.143409    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10-console-config\") pod \"console-5f8954cf7f-gnzjs\" (UID: \"8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10\") " pod="openshift-console/console-5f8954cf7f-gnzjs"
Jul 30 19:31:11.144065 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:11.144044    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10-service-ca\") pod \"console-5f8954cf7f-gnzjs\" (UID: \"8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10\") " pod="openshift-console/console-5f8954cf7f-gnzjs"
Jul 30 19:31:11.144229 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:11.144210    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10-console-config\") pod \"console-5f8954cf7f-gnzjs\" (UID: \"8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10\") " pod="openshift-console/console-5f8954cf7f-gnzjs"
Jul 30 19:31:11.144268 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:11.144239    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10-trusted-ca-bundle\") pod \"console-5f8954cf7f-gnzjs\" (UID: \"8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10\") " pod="openshift-console/console-5f8954cf7f-gnzjs"
Jul 30 19:31:11.144304 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:11.144239    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10-oauth-serving-cert\") pod \"console-5f8954cf7f-gnzjs\" (UID: \"8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10\") " pod="openshift-console/console-5f8954cf7f-gnzjs"
Jul 30 19:31:11.146267 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:11.146244    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10-console-oauth-config\") pod \"console-5f8954cf7f-gnzjs\" (UID: \"8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10\") " pod="openshift-console/console-5f8954cf7f-gnzjs"
Jul 30 19:31:11.146455 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:11.146436    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10-console-serving-cert\") pod \"console-5f8954cf7f-gnzjs\" (UID: \"8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10\") " pod="openshift-console/console-5f8954cf7f-gnzjs"
Jul 30 19:31:11.151506 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:11.151483    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2wx7\" (UniqueName: \"kubernetes.io/projected/8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10-kube-api-access-m2wx7\") pod \"console-5f8954cf7f-gnzjs\" (UID: \"8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10\") " pod="openshift-console/console-5f8954cf7f-gnzjs"
Jul 30 19:31:11.220907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:11.220870    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:31:11.244241 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:11.244213    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f8954cf7f-gnzjs"
Jul 30 19:31:11.385545 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:11.385473    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f8954cf7f-gnzjs"]
Jul 30 19:31:11.387827 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:31:11.387799    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a9c2ffe_2cb6_4381_b125_6b2e7cb26a10.slice/crio-8c85fd2de5647db0e074393da0d6ace76311e2cfc938dc9ec3f4eeb3629e04b4 WatchSource:0}: Error finding container 8c85fd2de5647db0e074393da0d6ace76311e2cfc938dc9ec3f4eeb3629e04b4: Status 404 returned error can't find the container with id 8c85fd2de5647db0e074393da0d6ace76311e2cfc938dc9ec3f4eeb3629e04b4
Jul 30 19:31:11.434769 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:11.434734    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f8954cf7f-gnzjs" event={"ID":"8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10","Type":"ContainerStarted","Data":"8c85fd2de5647db0e074393da0d6ace76311e2cfc938dc9ec3f4eeb3629e04b4"}
Jul 30 19:31:11.717349 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:11.717274    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:31:12.441653 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:12.441616    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f8954cf7f-gnzjs" event={"ID":"8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10","Type":"ContainerStarted","Data":"5302d562ba03374415f1b5daafd7f36f52445cfb621eb2867690ae84f3e3ae92"}
Jul 30 19:31:12.460117 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:12.460065    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f8954cf7f-gnzjs" podStartSLOduration=2.460049266 podStartE2EDuration="2.460049266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:31:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:31:12.458733559 +0000 UTC m=+400.423314633" watchObservedRunningTime="2026-07-30 19:31:12.460049266 +0000 UTC m=+400.424630347"
Jul 30 19:31:15.762203 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:15.762169    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:31:15.868487 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:15.868455    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:31:15.902654 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:15.902619    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rfbfp"]
Jul 30 19:31:15.906030 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:15.906005    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rfbfp"
Jul 30 19:31:15.908390 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:15.908359    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-55gh4\""
Jul 30 19:31:15.908527 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:15.908409    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 30 19:31:15.908527 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:15.908500    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 30 19:31:15.917186 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:15.917158    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rfbfp"]
Jul 30 19:31:15.943793 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:15.943759    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:31:16.091052 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:16.090933    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v27pv\" (UniqueName: \"kubernetes.io/projected/cd1b26ae-5a62-4ea7-8e70-2bdb1f908a24-kube-api-access-v27pv\") pod \"kuadrant-operator-catalog-rfbfp\" (UID: \"cd1b26ae-5a62-4ea7-8e70-2bdb1f908a24\") " pod="kuadrant-system/kuadrant-operator-catalog-rfbfp"
Jul 30 19:31:16.192031 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:16.191989    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v27pv\" (UniqueName: \"kubernetes.io/projected/cd1b26ae-5a62-4ea7-8e70-2bdb1f908a24-kube-api-access-v27pv\") pod \"kuadrant-operator-catalog-rfbfp\" (UID: \"cd1b26ae-5a62-4ea7-8e70-2bdb1f908a24\") " pod="kuadrant-system/kuadrant-operator-catalog-rfbfp"
Jul 30 19:31:16.199616 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:16.199576    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v27pv\" (UniqueName: \"kubernetes.io/projected/cd1b26ae-5a62-4ea7-8e70-2bdb1f908a24-kube-api-access-v27pv\") pod \"kuadrant-operator-catalog-rfbfp\" (UID: \"cd1b26ae-5a62-4ea7-8e70-2bdb1f908a24\") " pod="kuadrant-system/kuadrant-operator-catalog-rfbfp"
Jul 30 19:31:16.219679 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:16.219642    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rfbfp"
Jul 30 19:31:16.258618 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:16.258583    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rfbfp"]
Jul 30 19:31:16.355552 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:16.355523    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rfbfp"]
Jul 30 19:31:16.358116 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:31:16.358086    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd1b26ae_5a62_4ea7_8e70_2bdb1f908a24.slice/crio-f59c07991a1ffc03fffcd3c640350fb0ebef7ef39663a83b8be62e0b7cbf341f WatchSource:0}: Error finding container f59c07991a1ffc03fffcd3c640350fb0ebef7ef39663a83b8be62e0b7cbf341f: Status 404 returned error can't find the container with id f59c07991a1ffc03fffcd3c640350fb0ebef7ef39663a83b8be62e0b7cbf341f
Jul 30 19:31:16.463687 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:16.463655    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rfbfp" event={"ID":"cd1b26ae-5a62-4ea7-8e70-2bdb1f908a24","Type":"ContainerStarted","Data":"f59c07991a1ffc03fffcd3c640350fb0ebef7ef39663a83b8be62e0b7cbf341f"}
Jul 30 19:31:16.468322 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:16.468294    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rmxcj"]
Jul 30 19:31:16.472855 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:16.472838    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rmxcj"
Jul 30 19:31:16.479773 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:16.479744    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rmxcj"]
Jul 30 19:31:16.595467 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:16.595424    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w8mm\" (UniqueName: \"kubernetes.io/projected/c2145a09-d30d-4832-9b2d-40f1beebb16c-kube-api-access-2w8mm\") pod \"kuadrant-operator-catalog-rmxcj\" (UID: \"c2145a09-d30d-4832-9b2d-40f1beebb16c\") " pod="kuadrant-system/kuadrant-operator-catalog-rmxcj"
Jul 30 19:31:16.696240 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:16.696137    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w8mm\" (UniqueName: \"kubernetes.io/projected/c2145a09-d30d-4832-9b2d-40f1beebb16c-kube-api-access-2w8mm\") pod \"kuadrant-operator-catalog-rmxcj\" (UID: \"c2145a09-d30d-4832-9b2d-40f1beebb16c\") " pod="kuadrant-system/kuadrant-operator-catalog-rmxcj"
Jul 30 19:31:16.704465 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:16.704434    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w8mm\" (UniqueName: \"kubernetes.io/projected/c2145a09-d30d-4832-9b2d-40f1beebb16c-kube-api-access-2w8mm\") pod \"kuadrant-operator-catalog-rmxcj\" (UID: \"c2145a09-d30d-4832-9b2d-40f1beebb16c\") " pod="kuadrant-system/kuadrant-operator-catalog-rmxcj"
Jul 30 19:31:16.784438 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:16.784401    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rmxcj"
Jul 30 19:31:16.924525 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:16.924498    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rmxcj"]
Jul 30 19:31:16.926224 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:31:16.926201    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2145a09_d30d_4832_9b2d_40f1beebb16c.slice/crio-a84a3834511548596c05b682dc39eb98fdecb8c2c0ffbaf078350066b6738334 WatchSource:0}: Error finding container a84a3834511548596c05b682dc39eb98fdecb8c2c0ffbaf078350066b6738334: Status 404 returned error can't find the container with id a84a3834511548596c05b682dc39eb98fdecb8c2c0ffbaf078350066b6738334
Jul 30 19:31:17.470539 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:17.470508    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rmxcj" event={"ID":"c2145a09-d30d-4832-9b2d-40f1beebb16c","Type":"ContainerStarted","Data":"a84a3834511548596c05b682dc39eb98fdecb8c2c0ffbaf078350066b6738334"}
Jul 30 19:31:19.483572 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:19.483529    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rfbfp" event={"ID":"cd1b26ae-5a62-4ea7-8e70-2bdb1f908a24","Type":"ContainerStarted","Data":"ed1f4a74dc314c480bafd1e9a15ccfa63c75f1ee6ffd411c4764a7e28765202a"}
Jul 30 19:31:19.484074 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:19.483583    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-rfbfp" podUID="cd1b26ae-5a62-4ea7-8e70-2bdb1f908a24" containerName="registry-server" containerID="cri-o://ed1f4a74dc314c480bafd1e9a15ccfa63c75f1ee6ffd411c4764a7e28765202a" gracePeriod=2
Jul 30 19:31:19.485123 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:19.485089    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rmxcj" event={"ID":"c2145a09-d30d-4832-9b2d-40f1beebb16c","Type":"ContainerStarted","Data":"7bc7651cbab6eca1b9e0f35dbe4569eabeda7ff17a0e51b383779b7660fe529a"}
Jul 30 19:31:19.500679 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:19.500633    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-rfbfp" podStartSLOduration=2.260374483 podStartE2EDuration="4.50061683s" totalImagesPullingTime="2.240242347s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:31:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:31:19.499353269 +0000 UTC m=+407.463934352" watchObservedRunningTime="2026-07-30 19:31:19.50061683 +0000 UTC m=+407.465197911"
Jul 30 19:31:19.520176 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:19.520120    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-rmxcj" podStartSLOduration=1.8449776199999999 podStartE2EDuration="3.520103343s" totalImagesPullingTime="1.675125723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:31:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:31:19.518549594 +0000 UTC m=+407.483130685" watchObservedRunningTime="2026-07-30 19:31:19.520103343 +0000 UTC m=+407.484684424"
Jul 30 19:31:19.729926 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:19.729901    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rfbfp"
Jul 30 19:31:19.824374 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:19.824280    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v27pv\" (UniqueName: \"kubernetes.io/projected/cd1b26ae-5a62-4ea7-8e70-2bdb1f908a24-kube-api-access-v27pv\") pod \"cd1b26ae-5a62-4ea7-8e70-2bdb1f908a24\" (UID: \"cd1b26ae-5a62-4ea7-8e70-2bdb1f908a24\") "
Jul 30 19:31:19.826558 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:19.826524    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd1b26ae-5a62-4ea7-8e70-2bdb1f908a24-kube-api-access-v27pv" (OuterVolumeSpecName: "kube-api-access-v27pv") pod "cd1b26ae-5a62-4ea7-8e70-2bdb1f908a24" (UID: "cd1b26ae-5a62-4ea7-8e70-2bdb1f908a24"). InnerVolumeSpecName "kube-api-access-v27pv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:31:19.925169 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:19.925124    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v27pv\" (UniqueName: \"kubernetes.io/projected/cd1b26ae-5a62-4ea7-8e70-2bdb1f908a24-kube-api-access-v27pv\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:31:20.491188 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:20.491152    2554 generic.go:356] "Generic (PLEG): container finished" podID="cd1b26ae-5a62-4ea7-8e70-2bdb1f908a24" containerID="ed1f4a74dc314c480bafd1e9a15ccfa63c75f1ee6ffd411c4764a7e28765202a" exitCode=0
Jul 30 19:31:20.491677 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:20.491221    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rfbfp"
Jul 30 19:31:20.491677 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:20.491239    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rfbfp" event={"ID":"cd1b26ae-5a62-4ea7-8e70-2bdb1f908a24","Type":"ContainerDied","Data":"ed1f4a74dc314c480bafd1e9a15ccfa63c75f1ee6ffd411c4764a7e28765202a"}
Jul 30 19:31:20.491677 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:20.491293    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rfbfp" event={"ID":"cd1b26ae-5a62-4ea7-8e70-2bdb1f908a24","Type":"ContainerDied","Data":"f59c07991a1ffc03fffcd3c640350fb0ebef7ef39663a83b8be62e0b7cbf341f"}
Jul 30 19:31:20.491677 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:20.491315    2554 scope.go:117] "RemoveContainer" containerID="ed1f4a74dc314c480bafd1e9a15ccfa63c75f1ee6ffd411c4764a7e28765202a"
Jul 30 19:31:20.500993 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:20.500973    2554 scope.go:117] "RemoveContainer" containerID="ed1f4a74dc314c480bafd1e9a15ccfa63c75f1ee6ffd411c4764a7e28765202a"
Jul 30 19:31:20.501270 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:31:20.501250    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed1f4a74dc314c480bafd1e9a15ccfa63c75f1ee6ffd411c4764a7e28765202a\": container with ID starting with ed1f4a74dc314c480bafd1e9a15ccfa63c75f1ee6ffd411c4764a7e28765202a not found: ID does not exist" containerID="ed1f4a74dc314c480bafd1e9a15ccfa63c75f1ee6ffd411c4764a7e28765202a"
Jul 30 19:31:20.501326 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:20.501278    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed1f4a74dc314c480bafd1e9a15ccfa63c75f1ee6ffd411c4764a7e28765202a"} err="failed to get container status \"ed1f4a74dc314c480bafd1e9a15ccfa63c75f1ee6ffd411c4764a7e28765202a\": rpc error: code = NotFound desc = could not find container \"ed1f4a74dc314c480bafd1e9a15ccfa63c75f1ee6ffd411c4764a7e28765202a\": container with ID starting with ed1f4a74dc314c480bafd1e9a15ccfa63c75f1ee6ffd411c4764a7e28765202a not found: ID does not exist"
Jul 30 19:31:20.511614 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:20.511582    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rfbfp"]
Jul 30 19:31:20.515596 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:20.515575    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rfbfp"]
Jul 30 19:31:21.245036 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:21.244995    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f8954cf7f-gnzjs"
Jul 30 19:31:21.245225 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:21.245058    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f8954cf7f-gnzjs"
Jul 30 19:31:21.249979 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:21.249931    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f8954cf7f-gnzjs"
Jul 30 19:31:21.501564 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:21.501486    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f8954cf7f-gnzjs"
Jul 30 19:31:21.550231 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:21.550191    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-869fdf5f7b-h9x47"]
Jul 30 19:31:21.652268 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:21.652233    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:31:21.753358 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:21.753256    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:31:21.803840 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:21.803812    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:31:22.486655 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:22.486620    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd1b26ae-5a62-4ea7-8e70-2bdb1f908a24" path="/var/lib/kubelet/pods/cd1b26ae-5a62-4ea7-8e70-2bdb1f908a24/volumes"
Jul 30 19:31:26.784848 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:26.784804    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-rmxcj"
Jul 30 19:31:26.785312 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:26.784874    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-rmxcj"
Jul 30 19:31:26.806910 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:26.806882    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-rmxcj"
Jul 30 19:31:27.549399 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:27.549371    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-rmxcj"
Jul 30 19:31:31.273971 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:31.273914    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr"]
Jul 30 19:31:31.274340 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:31.274309    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd1b26ae-5a62-4ea7-8e70-2bdb1f908a24" containerName="registry-server"
Jul 30 19:31:31.274340 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:31.274321    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd1b26ae-5a62-4ea7-8e70-2bdb1f908a24" containerName="registry-server"
Jul 30 19:31:31.274425 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:31.274387    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd1b26ae-5a62-4ea7-8e70-2bdb1f908a24" containerName="registry-server"
Jul 30 19:31:31.277516 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:31.277498    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr"
Jul 30 19:31:31.280666 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:31.280644    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gnqwl\""
Jul 30 19:31:31.289078 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:31.289051    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr"]
Jul 30 19:31:31.313780 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:31.313748    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32b10063-97db-47e5-bd85-d3dd6633e9f5-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr\" (UID: \"32b10063-97db-47e5-bd85-d3dd6633e9f5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr"
Jul 30 19:31:31.313969 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:31.313798    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32b10063-97db-47e5-bd85-d3dd6633e9f5-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr\" (UID: \"32b10063-97db-47e5-bd85-d3dd6633e9f5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr"
Jul 30 19:31:31.313969 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:31.313911    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7n52b\" (UniqueName: \"kubernetes.io/projected/32b10063-97db-47e5-bd85-d3dd6633e9f5-kube-api-access-7n52b\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr\" (UID: \"32b10063-97db-47e5-bd85-d3dd6633e9f5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr"
Jul 30 19:31:31.415319 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:31.415276    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7n52b\" (UniqueName: \"kubernetes.io/projected/32b10063-97db-47e5-bd85-d3dd6633e9f5-kube-api-access-7n52b\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr\" (UID: \"32b10063-97db-47e5-bd85-d3dd6633e9f5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr"
Jul 30 19:31:31.415507 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:31.415337    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32b10063-97db-47e5-bd85-d3dd6633e9f5-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr\" (UID: \"32b10063-97db-47e5-bd85-d3dd6633e9f5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr"
Jul 30 19:31:31.415507 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:31.415377    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32b10063-97db-47e5-bd85-d3dd6633e9f5-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr\" (UID: \"32b10063-97db-47e5-bd85-d3dd6633e9f5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr"
Jul 30 19:31:31.415771 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:31.415755    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32b10063-97db-47e5-bd85-d3dd6633e9f5-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr\" (UID: \"32b10063-97db-47e5-bd85-d3dd6633e9f5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr"
Jul 30 19:31:31.415815 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:31.415774    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32b10063-97db-47e5-bd85-d3dd6633e9f5-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr\" (UID: \"32b10063-97db-47e5-bd85-d3dd6633e9f5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr"
Jul 30 19:31:31.423972 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:31.423926    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7n52b\" (UniqueName: \"kubernetes.io/projected/32b10063-97db-47e5-bd85-d3dd6633e9f5-kube-api-access-7n52b\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr\" (UID: \"32b10063-97db-47e5-bd85-d3dd6633e9f5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr"
Jul 30 19:31:31.586683 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:31.586594    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr"
Jul 30 19:31:31.733566 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:31.733541    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr"]
Jul 30 19:31:31.735846 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:31:31.735815    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32b10063_97db_47e5_bd85_d3dd6633e9f5.slice/crio-50703dcb8352c049c6d38be7ffbafb19f59d544de1ac9df53ae907babd0e3bda WatchSource:0}: Error finding container 50703dcb8352c049c6d38be7ffbafb19f59d544de1ac9df53ae907babd0e3bda: Status 404 returned error can't find the container with id 50703dcb8352c049c6d38be7ffbafb19f59d544de1ac9df53ae907babd0e3bda
Jul 30 19:31:32.068985 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.068937    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx"]
Jul 30 19:31:32.072413 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.072395    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx"
Jul 30 19:31:32.080650 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.080617    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx"]
Jul 30 19:31:32.121293 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.121244    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a84b32c-a92e-45e6-ae5c-6a59c1618028-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx\" (UID: \"2a84b32c-a92e-45e6-ae5c-6a59c1618028\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx"
Jul 30 19:31:32.121508 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.121363    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a84b32c-a92e-45e6-ae5c-6a59c1618028-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx\" (UID: \"2a84b32c-a92e-45e6-ae5c-6a59c1618028\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx"
Jul 30 19:31:32.121508 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.121425    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4gwr\" (UniqueName: \"kubernetes.io/projected/2a84b32c-a92e-45e6-ae5c-6a59c1618028-kube-api-access-p4gwr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx\" (UID: \"2a84b32c-a92e-45e6-ae5c-6a59c1618028\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx"
Jul 30 19:31:32.222350 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.222307    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a84b32c-a92e-45e6-ae5c-6a59c1618028-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx\" (UID: \"2a84b32c-a92e-45e6-ae5c-6a59c1618028\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx"
Jul 30 19:31:32.222556 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.222380    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4gwr\" (UniqueName: \"kubernetes.io/projected/2a84b32c-a92e-45e6-ae5c-6a59c1618028-kube-api-access-p4gwr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx\" (UID: \"2a84b32c-a92e-45e6-ae5c-6a59c1618028\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx"
Jul 30 19:31:32.222556 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.222424    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a84b32c-a92e-45e6-ae5c-6a59c1618028-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx\" (UID: \"2a84b32c-a92e-45e6-ae5c-6a59c1618028\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx"
Jul 30 19:31:32.222789 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.222768    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a84b32c-a92e-45e6-ae5c-6a59c1618028-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx\" (UID: \"2a84b32c-a92e-45e6-ae5c-6a59c1618028\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx"
Jul 30 19:31:32.222789 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.222780    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a84b32c-a92e-45e6-ae5c-6a59c1618028-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx\" (UID: \"2a84b32c-a92e-45e6-ae5c-6a59c1618028\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx"
Jul 30 19:31:32.232074 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.232040    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4gwr\" (UniqueName: \"kubernetes.io/projected/2a84b32c-a92e-45e6-ae5c-6a59c1618028-kube-api-access-p4gwr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx\" (UID: \"2a84b32c-a92e-45e6-ae5c-6a59c1618028\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx"
Jul 30 19:31:32.382215 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.382139    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx"
Jul 30 19:31:32.522037 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.522013    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx"]
Jul 30 19:31:32.524765 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:31:32.524736    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a84b32c_a92e_45e6_ae5c_6a59c1618028.slice/crio-465793f61c352cb0eb68c3d646317d632465e7f6a4fb87f7cb3aaa4cbb5aaac9 WatchSource:0}: Error finding container 465793f61c352cb0eb68c3d646317d632465e7f6a4fb87f7cb3aaa4cbb5aaac9: Status 404 returned error can't find the container with id 465793f61c352cb0eb68c3d646317d632465e7f6a4fb87f7cb3aaa4cbb5aaac9
Jul 30 19:31:32.555393 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.555364    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx" event={"ID":"2a84b32c-a92e-45e6-ae5c-6a59c1618028","Type":"ContainerStarted","Data":"465793f61c352cb0eb68c3d646317d632465e7f6a4fb87f7cb3aaa4cbb5aaac9"}
Jul 30 19:31:32.556712 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.556685    2554 generic.go:356] "Generic (PLEG): container finished" podID="32b10063-97db-47e5-bd85-d3dd6633e9f5" containerID="25c2d28d0a3a8abbfb58626f79e846f58640abea774e0e6585c433e634ad591f" exitCode=0
Jul 30 19:31:32.556837 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.556754    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr" event={"ID":"32b10063-97db-47e5-bd85-d3dd6633e9f5","Type":"ContainerDied","Data":"25c2d28d0a3a8abbfb58626f79e846f58640abea774e0e6585c433e634ad591f"}
Jul 30 19:31:32.556837 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.556794    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr" event={"ID":"32b10063-97db-47e5-bd85-d3dd6633e9f5","Type":"ContainerStarted","Data":"50703dcb8352c049c6d38be7ffbafb19f59d544de1ac9df53ae907babd0e3bda"}
Jul 30 19:31:32.674472 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.674436    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4"]
Jul 30 19:31:32.677870 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.677850    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4"
Jul 30 19:31:32.690245 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.690210    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4"]
Jul 30 19:31:32.726971 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.726920    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0266cc8-8703-4da9-9029-2170fdf22e9d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4\" (UID: \"e0266cc8-8703-4da9-9029-2170fdf22e9d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4"
Jul 30 19:31:32.727152 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.726986    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd2df\" (UniqueName: \"kubernetes.io/projected/e0266cc8-8703-4da9-9029-2170fdf22e9d-kube-api-access-dd2df\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4\" (UID: \"e0266cc8-8703-4da9-9029-2170fdf22e9d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4"
Jul 30 19:31:32.727152 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.727029    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0266cc8-8703-4da9-9029-2170fdf22e9d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4\" (UID: \"e0266cc8-8703-4da9-9029-2170fdf22e9d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4"
Jul 30 19:31:32.828552 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.828498    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0266cc8-8703-4da9-9029-2170fdf22e9d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4\" (UID: \"e0266cc8-8703-4da9-9029-2170fdf22e9d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4"
Jul 30 19:31:32.828552 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.828553    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd2df\" (UniqueName: \"kubernetes.io/projected/e0266cc8-8703-4da9-9029-2170fdf22e9d-kube-api-access-dd2df\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4\" (UID: \"e0266cc8-8703-4da9-9029-2170fdf22e9d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4"
Jul 30 19:31:32.828816 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.828599    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0266cc8-8703-4da9-9029-2170fdf22e9d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4\" (UID: \"e0266cc8-8703-4da9-9029-2170fdf22e9d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4"
Jul 30 19:31:32.828918 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.828887    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0266cc8-8703-4da9-9029-2170fdf22e9d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4\" (UID: \"e0266cc8-8703-4da9-9029-2170fdf22e9d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4"
Jul 30 19:31:32.829058 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.829037    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0266cc8-8703-4da9-9029-2170fdf22e9d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4\" (UID: \"e0266cc8-8703-4da9-9029-2170fdf22e9d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4"
Jul 30 19:31:32.837248 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:32.837226    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd2df\" (UniqueName: \"kubernetes.io/projected/e0266cc8-8703-4da9-9029-2170fdf22e9d-kube-api-access-dd2df\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4\" (UID: \"e0266cc8-8703-4da9-9029-2170fdf22e9d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4"
Jul 30 19:31:33.005859 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:33.005821    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4"
Jul 30 19:31:33.074273 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:33.074229    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq"]
Jul 30 19:31:33.078777 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:33.078754    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq"
Jul 30 19:31:33.085640 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:33.085610    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq"]
Jul 30 19:31:33.131047 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:33.131013    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vncht\" (UniqueName: \"kubernetes.io/projected/29bcabf6-73c0-47a5-8d47-0a92f7a945f7-kube-api-access-vncht\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq\" (UID: \"29bcabf6-73c0-47a5-8d47-0a92f7a945f7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq"
Jul 30 19:31:33.131245 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:33.131179    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29bcabf6-73c0-47a5-8d47-0a92f7a945f7-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq\" (UID: \"29bcabf6-73c0-47a5-8d47-0a92f7a945f7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq"
Jul 30 19:31:33.131245 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:33.131222    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29bcabf6-73c0-47a5-8d47-0a92f7a945f7-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq\" (UID: \"29bcabf6-73c0-47a5-8d47-0a92f7a945f7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq"
Jul 30 19:31:33.154147 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:33.154113    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4"]
Jul 30 19:31:33.155962 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:31:33.155921    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0266cc8_8703_4da9_9029_2170fdf22e9d.slice/crio-2ecea60d506776c37cedd5f9518854f5a6f68fa2ba22912db6a6e70642090903 WatchSource:0}: Error finding container 2ecea60d506776c37cedd5f9518854f5a6f68fa2ba22912db6a6e70642090903: Status 404 returned error can't find the container with id 2ecea60d506776c37cedd5f9518854f5a6f68fa2ba22912db6a6e70642090903
Jul 30 19:31:33.232226 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:33.232193    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29bcabf6-73c0-47a5-8d47-0a92f7a945f7-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq\" (UID: \"29bcabf6-73c0-47a5-8d47-0a92f7a945f7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq"
Jul 30 19:31:33.232360 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:33.232246    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29bcabf6-73c0-47a5-8d47-0a92f7a945f7-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq\" (UID: \"29bcabf6-73c0-47a5-8d47-0a92f7a945f7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq"
Jul 30 19:31:33.232360 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:33.232321    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vncht\" (UniqueName: \"kubernetes.io/projected/29bcabf6-73c0-47a5-8d47-0a92f7a945f7-kube-api-access-vncht\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq\" (UID: \"29bcabf6-73c0-47a5-8d47-0a92f7a945f7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq"
Jul 30 19:31:33.232746 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:33.232688    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29bcabf6-73c0-47a5-8d47-0a92f7a945f7-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq\" (UID: \"29bcabf6-73c0-47a5-8d47-0a92f7a945f7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq"
Jul 30 19:31:33.232857 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:33.232833    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29bcabf6-73c0-47a5-8d47-0a92f7a945f7-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq\" (UID: \"29bcabf6-73c0-47a5-8d47-0a92f7a945f7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq"
Jul 30 19:31:33.241761 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:33.241729    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vncht\" (UniqueName: \"kubernetes.io/projected/29bcabf6-73c0-47a5-8d47-0a92f7a945f7-kube-api-access-vncht\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq\" (UID: \"29bcabf6-73c0-47a5-8d47-0a92f7a945f7\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq"
Jul 30 19:31:33.391100 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:33.391064    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq"
Jul 30 19:31:33.531231 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:33.531205    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq"]
Jul 30 19:31:33.533648 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:31:33.533619    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29bcabf6_73c0_47a5_8d47_0a92f7a945f7.slice/crio-be11d84a61f523dde923ea8d3971a37eab3d3a35c6712d85fcf79893cc42465e WatchSource:0}: Error finding container be11d84a61f523dde923ea8d3971a37eab3d3a35c6712d85fcf79893cc42465e: Status 404 returned error can't find the container with id be11d84a61f523dde923ea8d3971a37eab3d3a35c6712d85fcf79893cc42465e
Jul 30 19:31:33.563011 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:33.562980    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq" event={"ID":"29bcabf6-73c0-47a5-8d47-0a92f7a945f7","Type":"ContainerStarted","Data":"be11d84a61f523dde923ea8d3971a37eab3d3a35c6712d85fcf79893cc42465e"}
Jul 30 19:31:33.564785 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:33.564760    2554 generic.go:356] "Generic (PLEG): container finished" podID="32b10063-97db-47e5-bd85-d3dd6633e9f5" containerID="a14fddd18e50fe39db01ccb979768163e928a799ab397f6040e5a04ffd9838d2" exitCode=0
Jul 30 19:31:33.564920 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:33.564852    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr" event={"ID":"32b10063-97db-47e5-bd85-d3dd6633e9f5","Type":"ContainerDied","Data":"a14fddd18e50fe39db01ccb979768163e928a799ab397f6040e5a04ffd9838d2"}
Jul 30 19:31:33.566413 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:33.566392    2554 generic.go:356] "Generic (PLEG): container finished" podID="e0266cc8-8703-4da9-9029-2170fdf22e9d" containerID="88a419d9600d2edbebaa1ac8512cad37ef414d1fdca934bb73f4956f5da6ad0d" exitCode=0
Jul 30 19:31:33.566498 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:33.566462    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4" event={"ID":"e0266cc8-8703-4da9-9029-2170fdf22e9d","Type":"ContainerDied","Data":"88a419d9600d2edbebaa1ac8512cad37ef414d1fdca934bb73f4956f5da6ad0d"}
Jul 30 19:31:33.566498 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:33.566495    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4" event={"ID":"e0266cc8-8703-4da9-9029-2170fdf22e9d","Type":"ContainerStarted","Data":"2ecea60d506776c37cedd5f9518854f5a6f68fa2ba22912db6a6e70642090903"}
Jul 30 19:31:33.568090 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:33.568068    2554 generic.go:356] "Generic (PLEG): container finished" podID="2a84b32c-a92e-45e6-ae5c-6a59c1618028" containerID="56cfc46e1ed4db368627467696e0078e7299c865c6bab8d0ca0c068e1b4ee5d1" exitCode=0
Jul 30 19:31:33.568183 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:33.568153    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx" event={"ID":"2a84b32c-a92e-45e6-ae5c-6a59c1618028","Type":"ContainerDied","Data":"56cfc46e1ed4db368627467696e0078e7299c865c6bab8d0ca0c068e1b4ee5d1"}
Jul 30 19:31:34.577416 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:34.577383    2554 generic.go:356] "Generic (PLEG): container finished" podID="e0266cc8-8703-4da9-9029-2170fdf22e9d" containerID="8756f961dc5a6054e5cfb58f2a75ed3d60ae3e36559e773b529c0049acc8f527" exitCode=0
Jul 30 19:31:34.577851 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:34.577473    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4" event={"ID":"e0266cc8-8703-4da9-9029-2170fdf22e9d","Type":"ContainerDied","Data":"8756f961dc5a6054e5cfb58f2a75ed3d60ae3e36559e773b529c0049acc8f527"}
Jul 30 19:31:34.579094 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:34.579052    2554 generic.go:356] "Generic (PLEG): container finished" podID="29bcabf6-73c0-47a5-8d47-0a92f7a945f7" containerID="648ca50b86004899d1c4361a61c5f04bd2930c488013565615d4a63d0eddf24f" exitCode=0
Jul 30 19:31:34.579153 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:34.579135    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq" event={"ID":"29bcabf6-73c0-47a5-8d47-0a92f7a945f7","Type":"ContainerDied","Data":"648ca50b86004899d1c4361a61c5f04bd2930c488013565615d4a63d0eddf24f"}
Jul 30 19:31:34.581625 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:34.581552    2554 generic.go:356] "Generic (PLEG): container finished" podID="32b10063-97db-47e5-bd85-d3dd6633e9f5" containerID="01a2b95cf2fc5af03d469fc0adce9d56deea91d71317145cad31f6c4d3b0b4f0" exitCode=0
Jul 30 19:31:34.581625 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:34.581599    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr" event={"ID":"32b10063-97db-47e5-bd85-d3dd6633e9f5","Type":"ContainerDied","Data":"01a2b95cf2fc5af03d469fc0adce9d56deea91d71317145cad31f6c4d3b0b4f0"}
Jul 30 19:31:35.590565 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:35.590533    2554 generic.go:356] "Generic (PLEG): container finished" podID="e0266cc8-8703-4da9-9029-2170fdf22e9d" containerID="f9c3e20d3ca2987f1f9aa8c1c4e88dab79ddbe56df5c244e244fc4120a43a7b1" exitCode=0
Jul 30 19:31:35.591027 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:35.590614    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4" event={"ID":"e0266cc8-8703-4da9-9029-2170fdf22e9d","Type":"ContainerDied","Data":"f9c3e20d3ca2987f1f9aa8c1c4e88dab79ddbe56df5c244e244fc4120a43a7b1"}
Jul 30 19:31:35.592551 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:35.592526    2554 generic.go:356] "Generic (PLEG): container finished" podID="2a84b32c-a92e-45e6-ae5c-6a59c1618028" containerID="333fb192483e9f248eb8ca4be4048ed4c36d2f91887760c1250e296bf55f83cb" exitCode=0
Jul 30 19:31:35.592689 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:35.592595    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx" event={"ID":"2a84b32c-a92e-45e6-ae5c-6a59c1618028","Type":"ContainerDied","Data":"333fb192483e9f248eb8ca4be4048ed4c36d2f91887760c1250e296bf55f83cb"}
Jul 30 19:31:35.731935 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:35.731906    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr"
Jul 30 19:31:35.757739 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:35.757708    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7n52b\" (UniqueName: \"kubernetes.io/projected/32b10063-97db-47e5-bd85-d3dd6633e9f5-kube-api-access-7n52b\") pod \"32b10063-97db-47e5-bd85-d3dd6633e9f5\" (UID: \"32b10063-97db-47e5-bd85-d3dd6633e9f5\") "
Jul 30 19:31:35.757925 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:35.757759    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32b10063-97db-47e5-bd85-d3dd6633e9f5-util\") pod \"32b10063-97db-47e5-bd85-d3dd6633e9f5\" (UID: \"32b10063-97db-47e5-bd85-d3dd6633e9f5\") "
Jul 30 19:31:35.757925 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:35.757917    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32b10063-97db-47e5-bd85-d3dd6633e9f5-bundle\") pod \"32b10063-97db-47e5-bd85-d3dd6633e9f5\" (UID: \"32b10063-97db-47e5-bd85-d3dd6633e9f5\") "
Jul 30 19:31:35.758685 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:35.758397    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32b10063-97db-47e5-bd85-d3dd6633e9f5-bundle" (OuterVolumeSpecName: "bundle") pod "32b10063-97db-47e5-bd85-d3dd6633e9f5" (UID: "32b10063-97db-47e5-bd85-d3dd6633e9f5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:31:35.760061 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:35.760028    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32b10063-97db-47e5-bd85-d3dd6633e9f5-kube-api-access-7n52b" (OuterVolumeSpecName: "kube-api-access-7n52b") pod "32b10063-97db-47e5-bd85-d3dd6633e9f5" (UID: "32b10063-97db-47e5-bd85-d3dd6633e9f5"). InnerVolumeSpecName "kube-api-access-7n52b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:31:35.765906 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:35.765873    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32b10063-97db-47e5-bd85-d3dd6633e9f5-util" (OuterVolumeSpecName: "util") pod "32b10063-97db-47e5-bd85-d3dd6633e9f5" (UID: "32b10063-97db-47e5-bd85-d3dd6633e9f5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:31:35.859397 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:35.859363    2554 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/32b10063-97db-47e5-bd85-d3dd6633e9f5-bundle\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:31:35.859397 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:35.859391    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7n52b\" (UniqueName: \"kubernetes.io/projected/32b10063-97db-47e5-bd85-d3dd6633e9f5-kube-api-access-7n52b\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:31:35.859397 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:35.859401    2554 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/32b10063-97db-47e5-bd85-d3dd6633e9f5-util\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:31:36.600415 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:36.600378    2554 generic.go:356] "Generic (PLEG): container finished" podID="2a84b32c-a92e-45e6-ae5c-6a59c1618028" containerID="fe1fdfda09420c4b781ccfe459e3c80079a92f8e4ff55360bd309ea9c79732ef" exitCode=0
Jul 30 19:31:36.600888 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:36.600423    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx" event={"ID":"2a84b32c-a92e-45e6-ae5c-6a59c1618028","Type":"ContainerDied","Data":"fe1fdfda09420c4b781ccfe459e3c80079a92f8e4ff55360bd309ea9c79732ef"}
Jul 30 19:31:36.602248 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:36.602224    2554 generic.go:356] "Generic (PLEG): container finished" podID="29bcabf6-73c0-47a5-8d47-0a92f7a945f7" containerID="fe3d774d13c4aaaa7012c49d0225a6514ac2e9ae125f46fc3f8e2466900863f9" exitCode=0
Jul 30 19:31:36.602376 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:36.602267    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq" event={"ID":"29bcabf6-73c0-47a5-8d47-0a92f7a945f7","Type":"ContainerDied","Data":"fe3d774d13c4aaaa7012c49d0225a6514ac2e9ae125f46fc3f8e2466900863f9"}
Jul 30 19:31:36.604259 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:36.604235    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr"
Jul 30 19:31:36.604351 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:36.604263    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr" event={"ID":"32b10063-97db-47e5-bd85-d3dd6633e9f5","Type":"ContainerDied","Data":"50703dcb8352c049c6d38be7ffbafb19f59d544de1ac9df53ae907babd0e3bda"}
Jul 30 19:31:36.604351 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:36.604299    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="50703dcb8352c049c6d38be7ffbafb19f59d544de1ac9df53ae907babd0e3bda"
Jul 30 19:31:36.744937 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:36.744907    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4"
Jul 30 19:31:36.767327 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:36.767286    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0266cc8-8703-4da9-9029-2170fdf22e9d-bundle\") pod \"e0266cc8-8703-4da9-9029-2170fdf22e9d\" (UID: \"e0266cc8-8703-4da9-9029-2170fdf22e9d\") "
Jul 30 19:31:36.767490 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:36.767398    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0266cc8-8703-4da9-9029-2170fdf22e9d-util\") pod \"e0266cc8-8703-4da9-9029-2170fdf22e9d\" (UID: \"e0266cc8-8703-4da9-9029-2170fdf22e9d\") "
Jul 30 19:31:36.767490 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:36.767439    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dd2df\" (UniqueName: \"kubernetes.io/projected/e0266cc8-8703-4da9-9029-2170fdf22e9d-kube-api-access-dd2df\") pod \"e0266cc8-8703-4da9-9029-2170fdf22e9d\" (UID: \"e0266cc8-8703-4da9-9029-2170fdf22e9d\") "
Jul 30 19:31:36.768531 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:36.768480    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0266cc8-8703-4da9-9029-2170fdf22e9d-bundle" (OuterVolumeSpecName: "bundle") pod "e0266cc8-8703-4da9-9029-2170fdf22e9d" (UID: "e0266cc8-8703-4da9-9029-2170fdf22e9d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:31:36.770117 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:36.770083    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0266cc8-8703-4da9-9029-2170fdf22e9d-kube-api-access-dd2df" (OuterVolumeSpecName: "kube-api-access-dd2df") pod "e0266cc8-8703-4da9-9029-2170fdf22e9d" (UID: "e0266cc8-8703-4da9-9029-2170fdf22e9d"). InnerVolumeSpecName "kube-api-access-dd2df". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:31:36.775003 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:36.774968    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0266cc8-8703-4da9-9029-2170fdf22e9d-util" (OuterVolumeSpecName: "util") pod "e0266cc8-8703-4da9-9029-2170fdf22e9d" (UID: "e0266cc8-8703-4da9-9029-2170fdf22e9d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:31:36.868720 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:36.868680    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dd2df\" (UniqueName: \"kubernetes.io/projected/e0266cc8-8703-4da9-9029-2170fdf22e9d-kube-api-access-dd2df\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:31:36.868720 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:36.868713    2554 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0266cc8-8703-4da9-9029-2170fdf22e9d-bundle\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:31:36.868720 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:36.868728    2554 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0266cc8-8703-4da9-9029-2170fdf22e9d-util\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:31:37.611658 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:37.611621    2554 generic.go:356] "Generic (PLEG): container finished" podID="29bcabf6-73c0-47a5-8d47-0a92f7a945f7" containerID="d2329b44e5e4955a69b88f580fd451b5bd41fff1e163db2be609db7e02ae7af8" exitCode=0
Jul 30 19:31:37.612105 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:37.611678    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq" event={"ID":"29bcabf6-73c0-47a5-8d47-0a92f7a945f7","Type":"ContainerDied","Data":"d2329b44e5e4955a69b88f580fd451b5bd41fff1e163db2be609db7e02ae7af8"}
Jul 30 19:31:37.613559 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:37.613540    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4"
Jul 30 19:31:37.613658 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:37.613577    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4" event={"ID":"e0266cc8-8703-4da9-9029-2170fdf22e9d","Type":"ContainerDied","Data":"2ecea60d506776c37cedd5f9518854f5a6f68fa2ba22912db6a6e70642090903"}
Jul 30 19:31:37.613658 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:37.613616    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ecea60d506776c37cedd5f9518854f5a6f68fa2ba22912db6a6e70642090903"
Jul 30 19:31:37.747112 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:37.747084    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx"
Jul 30 19:31:37.777032 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:37.777004    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a84b32c-a92e-45e6-ae5c-6a59c1618028-util\") pod \"2a84b32c-a92e-45e6-ae5c-6a59c1618028\" (UID: \"2a84b32c-a92e-45e6-ae5c-6a59c1618028\") "
Jul 30 19:31:37.777187 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:37.777041    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4gwr\" (UniqueName: \"kubernetes.io/projected/2a84b32c-a92e-45e6-ae5c-6a59c1618028-kube-api-access-p4gwr\") pod \"2a84b32c-a92e-45e6-ae5c-6a59c1618028\" (UID: \"2a84b32c-a92e-45e6-ae5c-6a59c1618028\") "
Jul 30 19:31:37.777187 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:37.777096    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a84b32c-a92e-45e6-ae5c-6a59c1618028-bundle\") pod \"2a84b32c-a92e-45e6-ae5c-6a59c1618028\" (UID: \"2a84b32c-a92e-45e6-ae5c-6a59c1618028\") "
Jul 30 19:31:37.777617 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:37.777587    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a84b32c-a92e-45e6-ae5c-6a59c1618028-bundle" (OuterVolumeSpecName: "bundle") pod "2a84b32c-a92e-45e6-ae5c-6a59c1618028" (UID: "2a84b32c-a92e-45e6-ae5c-6a59c1618028"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:31:37.779253 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:37.779228    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a84b32c-a92e-45e6-ae5c-6a59c1618028-kube-api-access-p4gwr" (OuterVolumeSpecName: "kube-api-access-p4gwr") pod "2a84b32c-a92e-45e6-ae5c-6a59c1618028" (UID: "2a84b32c-a92e-45e6-ae5c-6a59c1618028"). InnerVolumeSpecName "kube-api-access-p4gwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:31:37.786674 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:37.786592    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a84b32c-a92e-45e6-ae5c-6a59c1618028-util" (OuterVolumeSpecName: "util") pod "2a84b32c-a92e-45e6-ae5c-6a59c1618028" (UID: "2a84b32c-a92e-45e6-ae5c-6a59c1618028"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:31:37.878051 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:37.878013    2554 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a84b32c-a92e-45e6-ae5c-6a59c1618028-bundle\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:31:37.878051 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:37.878041    2554 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a84b32c-a92e-45e6-ae5c-6a59c1618028-util\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:31:37.878051 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:37.878051    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4gwr\" (UniqueName: \"kubernetes.io/projected/2a84b32c-a92e-45e6-ae5c-6a59c1618028-kube-api-access-p4gwr\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:31:38.620852 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:38.620823    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx"
Jul 30 19:31:38.621357 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:38.620891    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx" event={"ID":"2a84b32c-a92e-45e6-ae5c-6a59c1618028","Type":"ContainerDied","Data":"465793f61c352cb0eb68c3d646317d632465e7f6a4fb87f7cb3aaa4cbb5aaac9"}
Jul 30 19:31:38.621357 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:38.620944    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="465793f61c352cb0eb68c3d646317d632465e7f6a4fb87f7cb3aaa4cbb5aaac9"
Jul 30 19:31:38.754257 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:38.754185    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq"
Jul 30 19:31:38.786700 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:38.786664    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vncht\" (UniqueName: \"kubernetes.io/projected/29bcabf6-73c0-47a5-8d47-0a92f7a945f7-kube-api-access-vncht\") pod \"29bcabf6-73c0-47a5-8d47-0a92f7a945f7\" (UID: \"29bcabf6-73c0-47a5-8d47-0a92f7a945f7\") "
Jul 30 19:31:38.787224 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:38.786733    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29bcabf6-73c0-47a5-8d47-0a92f7a945f7-util\") pod \"29bcabf6-73c0-47a5-8d47-0a92f7a945f7\" (UID: \"29bcabf6-73c0-47a5-8d47-0a92f7a945f7\") "
Jul 30 19:31:38.787224 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:38.786777    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29bcabf6-73c0-47a5-8d47-0a92f7a945f7-bundle\") pod \"29bcabf6-73c0-47a5-8d47-0a92f7a945f7\" (UID: \"29bcabf6-73c0-47a5-8d47-0a92f7a945f7\") "
Jul 30 19:31:38.787554 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:38.787496    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29bcabf6-73c0-47a5-8d47-0a92f7a945f7-bundle" (OuterVolumeSpecName: "bundle") pod "29bcabf6-73c0-47a5-8d47-0a92f7a945f7" (UID: "29bcabf6-73c0-47a5-8d47-0a92f7a945f7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:31:38.789151 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:38.789118    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29bcabf6-73c0-47a5-8d47-0a92f7a945f7-kube-api-access-vncht" (OuterVolumeSpecName: "kube-api-access-vncht") pod "29bcabf6-73c0-47a5-8d47-0a92f7a945f7" (UID: "29bcabf6-73c0-47a5-8d47-0a92f7a945f7"). InnerVolumeSpecName "kube-api-access-vncht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:31:38.792530 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:38.792502    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29bcabf6-73c0-47a5-8d47-0a92f7a945f7-util" (OuterVolumeSpecName: "util") pod "29bcabf6-73c0-47a5-8d47-0a92f7a945f7" (UID: "29bcabf6-73c0-47a5-8d47-0a92f7a945f7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:31:38.888052 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:38.888012    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vncht\" (UniqueName: \"kubernetes.io/projected/29bcabf6-73c0-47a5-8d47-0a92f7a945f7-kube-api-access-vncht\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:31:38.888052 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:38.888045    2554 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29bcabf6-73c0-47a5-8d47-0a92f7a945f7-util\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:31:38.888052 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:38.888055    2554 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29bcabf6-73c0-47a5-8d47-0a92f7a945f7-bundle\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:31:39.628013 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:39.627976    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq" event={"ID":"29bcabf6-73c0-47a5-8d47-0a92f7a945f7","Type":"ContainerDied","Data":"be11d84a61f523dde923ea8d3971a37eab3d3a35c6712d85fcf79893cc42465e"}
Jul 30 19:31:39.628411 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:39.628024    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq"
Jul 30 19:31:39.628411 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:39.628030    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="be11d84a61f523dde923ea8d3971a37eab3d3a35c6712d85fcf79893cc42465e"
Jul 30 19:31:45.045638 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:45.045601    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:31:46.572467 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:46.572425    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-869fdf5f7b-h9x47" podUID="539b3bf5-a852-4029-8107-2d89ee7d6e5b" containerName="console" containerID="cri-o://82adce2724a2f1c30bd318873c36ef2e4146fcce1295deaf1dffee7ae526e661" gracePeriod=15
Jul 30 19:31:46.831102 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:46.831043    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:31:46.959760 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:46.959722    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/539b3bf5-a852-4029-8107-2d89ee7d6e5b-trusted-ca-bundle\") pod \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\" (UID: \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\") "
Jul 30 19:31:46.959760 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:46.959769    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/539b3bf5-a852-4029-8107-2d89ee7d6e5b-console-serving-cert\") pod \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\" (UID: \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\") "
Jul 30 19:31:46.960013 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:46.959790    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/539b3bf5-a852-4029-8107-2d89ee7d6e5b-console-config\") pod \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\" (UID: \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\") "
Jul 30 19:31:46.960013 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:46.959989    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/539b3bf5-a852-4029-8107-2d89ee7d6e5b-console-oauth-config\") pod \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\" (UID: \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\") "
Jul 30 19:31:46.960120 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:46.960029    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtfqx\" (UniqueName: \"kubernetes.io/projected/539b3bf5-a852-4029-8107-2d89ee7d6e5b-kube-api-access-dtfqx\") pod \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\" (UID: \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\") "
Jul 30 19:31:46.960120 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:46.960089    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/539b3bf5-a852-4029-8107-2d89ee7d6e5b-oauth-serving-cert\") pod \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\" (UID: \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\") "
Jul 30 19:31:46.960120 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:46.960114    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/539b3bf5-a852-4029-8107-2d89ee7d6e5b-service-ca\") pod \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\" (UID: \"539b3bf5-a852-4029-8107-2d89ee7d6e5b\") "
Jul 30 19:31:46.960280 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:46.960195    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/539b3bf5-a852-4029-8107-2d89ee7d6e5b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "539b3bf5-a852-4029-8107-2d89ee7d6e5b" (UID: "539b3bf5-a852-4029-8107-2d89ee7d6e5b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 19:31:46.960280 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:46.960267    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/539b3bf5-a852-4029-8107-2d89ee7d6e5b-console-config" (OuterVolumeSpecName: "console-config") pod "539b3bf5-a852-4029-8107-2d89ee7d6e5b" (UID: "539b3bf5-a852-4029-8107-2d89ee7d6e5b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 19:31:46.960565 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:46.960544    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/539b3bf5-a852-4029-8107-2d89ee7d6e5b-service-ca" (OuterVolumeSpecName: "service-ca") pod "539b3bf5-a852-4029-8107-2d89ee7d6e5b" (UID: "539b3bf5-a852-4029-8107-2d89ee7d6e5b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 19:31:46.960656 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:46.960553    2554 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/539b3bf5-a852-4029-8107-2d89ee7d6e5b-trusted-ca-bundle\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:31:46.960656 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:46.960581    2554 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/539b3bf5-a852-4029-8107-2d89ee7d6e5b-console-config\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:31:46.960656 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:46.960540    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/539b3bf5-a852-4029-8107-2d89ee7d6e5b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "539b3bf5-a852-4029-8107-2d89ee7d6e5b" (UID: "539b3bf5-a852-4029-8107-2d89ee7d6e5b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 19:31:46.962052 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:46.962022    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/539b3bf5-a852-4029-8107-2d89ee7d6e5b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "539b3bf5-a852-4029-8107-2d89ee7d6e5b" (UID: "539b3bf5-a852-4029-8107-2d89ee7d6e5b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 19:31:46.962162 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:46.962145    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/539b3bf5-a852-4029-8107-2d89ee7d6e5b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "539b3bf5-a852-4029-8107-2d89ee7d6e5b" (UID: "539b3bf5-a852-4029-8107-2d89ee7d6e5b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 19:31:46.962361 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:46.962340    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/539b3bf5-a852-4029-8107-2d89ee7d6e5b-kube-api-access-dtfqx" (OuterVolumeSpecName: "kube-api-access-dtfqx") pod "539b3bf5-a852-4029-8107-2d89ee7d6e5b" (UID: "539b3bf5-a852-4029-8107-2d89ee7d6e5b"). InnerVolumeSpecName "kube-api-access-dtfqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:31:47.061154 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.061116    2554 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/539b3bf5-a852-4029-8107-2d89ee7d6e5b-console-oauth-config\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:31:47.061154 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.061158    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtfqx\" (UniqueName: \"kubernetes.io/projected/539b3bf5-a852-4029-8107-2d89ee7d6e5b-kube-api-access-dtfqx\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:31:47.061385 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.061168    2554 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/539b3bf5-a852-4029-8107-2d89ee7d6e5b-oauth-serving-cert\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:31:47.061385 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.061177    2554 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/539b3bf5-a852-4029-8107-2d89ee7d6e5b-service-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:31:47.061385 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.061186    2554 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/539b3bf5-a852-4029-8107-2d89ee7d6e5b-console-serving-cert\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:31:47.670290 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.670255    2554 generic.go:356] "Generic (PLEG): container finished" podID="539b3bf5-a852-4029-8107-2d89ee7d6e5b" containerID="82adce2724a2f1c30bd318873c36ef2e4146fcce1295deaf1dffee7ae526e661" exitCode=0
Jul 30 19:31:47.670750 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.670313    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-869fdf5f7b-h9x47"
Jul 30 19:31:47.670750 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.670348    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-869fdf5f7b-h9x47" event={"ID":"539b3bf5-a852-4029-8107-2d89ee7d6e5b","Type":"ContainerDied","Data":"82adce2724a2f1c30bd318873c36ef2e4146fcce1295deaf1dffee7ae526e661"}
Jul 30 19:31:47.670750 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.670404    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-869fdf5f7b-h9x47" event={"ID":"539b3bf5-a852-4029-8107-2d89ee7d6e5b","Type":"ContainerDied","Data":"34e8cef42688fed453fa7c56a4a4397c36f9ce0e81177ad5c00e72fe9c35a63b"}
Jul 30 19:31:47.670750 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.670427    2554 scope.go:117] "RemoveContainer" containerID="82adce2724a2f1c30bd318873c36ef2e4146fcce1295deaf1dffee7ae526e661"
Jul 30 19:31:47.674526 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.674491    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-d9nzc"]
Jul 30 19:31:47.674907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.674845    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a84b32c-a92e-45e6-ae5c-6a59c1618028" containerName="util"
Jul 30 19:31:47.674907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.674858    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a84b32c-a92e-45e6-ae5c-6a59c1618028" containerName="util"
Jul 30 19:31:47.674907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.674868    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29bcabf6-73c0-47a5-8d47-0a92f7a945f7" containerName="util"
Jul 30 19:31:47.674907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.674874    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="29bcabf6-73c0-47a5-8d47-0a92f7a945f7" containerName="util"
Jul 30 19:31:47.674907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.674884    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="539b3bf5-a852-4029-8107-2d89ee7d6e5b" containerName="console"
Jul 30 19:31:47.674907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.674889    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="539b3bf5-a852-4029-8107-2d89ee7d6e5b" containerName="console"
Jul 30 19:31:47.674907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.674896    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a84b32c-a92e-45e6-ae5c-6a59c1618028" containerName="pull"
Jul 30 19:31:47.674907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.674901    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a84b32c-a92e-45e6-ae5c-6a59c1618028" containerName="pull"
Jul 30 19:31:47.674907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.674907    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32b10063-97db-47e5-bd85-d3dd6633e9f5" containerName="pull"
Jul 30 19:31:47.674907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.674912    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="32b10063-97db-47e5-bd85-d3dd6633e9f5" containerName="pull"
Jul 30 19:31:47.675335 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.674918    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0266cc8-8703-4da9-9029-2170fdf22e9d" containerName="pull"
Jul 30 19:31:47.675335 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.674923    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0266cc8-8703-4da9-9029-2170fdf22e9d" containerName="pull"
Jul 30 19:31:47.675335 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.674930    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0266cc8-8703-4da9-9029-2170fdf22e9d" containerName="util"
Jul 30 19:31:47.675335 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.674936    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0266cc8-8703-4da9-9029-2170fdf22e9d" containerName="util"
Jul 30 19:31:47.675335 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.674947    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0266cc8-8703-4da9-9029-2170fdf22e9d" containerName="extract"
Jul 30 19:31:47.675335 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.674967    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0266cc8-8703-4da9-9029-2170fdf22e9d" containerName="extract"
Jul 30 19:31:47.675335 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.674974    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a84b32c-a92e-45e6-ae5c-6a59c1618028" containerName="extract"
Jul 30 19:31:47.675335 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.674979    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a84b32c-a92e-45e6-ae5c-6a59c1618028" containerName="extract"
Jul 30 19:31:47.675335 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.674989    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32b10063-97db-47e5-bd85-d3dd6633e9f5" containerName="util"
Jul 30 19:31:47.675335 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.674993    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="32b10063-97db-47e5-bd85-d3dd6633e9f5" containerName="util"
Jul 30 19:31:47.675335 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.674999    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32b10063-97db-47e5-bd85-d3dd6633e9f5" containerName="extract"
Jul 30 19:31:47.675335 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.675004    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="32b10063-97db-47e5-bd85-d3dd6633e9f5" containerName="extract"
Jul 30 19:31:47.675335 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.675012    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29bcabf6-73c0-47a5-8d47-0a92f7a945f7" containerName="pull"
Jul 30 19:31:47.675335 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.675017    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="29bcabf6-73c0-47a5-8d47-0a92f7a945f7" containerName="pull"
Jul 30 19:31:47.675335 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.675026    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29bcabf6-73c0-47a5-8d47-0a92f7a945f7" containerName="extract"
Jul 30 19:31:47.675335 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.675031    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="29bcabf6-73c0-47a5-8d47-0a92f7a945f7" containerName="extract"
Jul 30 19:31:47.675335 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.675087    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="539b3bf5-a852-4029-8107-2d89ee7d6e5b" containerName="console"
Jul 30 19:31:47.675335 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.675096    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="29bcabf6-73c0-47a5-8d47-0a92f7a945f7" containerName="extract"
Jul 30 19:31:47.675335 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.675104    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="32b10063-97db-47e5-bd85-d3dd6633e9f5" containerName="extract"
Jul 30 19:31:47.675335 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.675109    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a84b32c-a92e-45e6-ae5c-6a59c1618028" containerName="extract"
Jul 30 19:31:47.675335 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.675115    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0266cc8-8703-4da9-9029-2170fdf22e9d" containerName="extract"
Jul 30 19:31:47.679523 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.679506    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d9nzc"
Jul 30 19:31:47.680840 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.680812    2554 scope.go:117] "RemoveContainer" containerID="82adce2724a2f1c30bd318873c36ef2e4146fcce1295deaf1dffee7ae526e661"
Jul 30 19:31:47.681149 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:31:47.681132    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82adce2724a2f1c30bd318873c36ef2e4146fcce1295deaf1dffee7ae526e661\": container with ID starting with 82adce2724a2f1c30bd318873c36ef2e4146fcce1295deaf1dffee7ae526e661 not found: ID does not exist" containerID="82adce2724a2f1c30bd318873c36ef2e4146fcce1295deaf1dffee7ae526e661"
Jul 30 19:31:47.681194 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.681160    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82adce2724a2f1c30bd318873c36ef2e4146fcce1295deaf1dffee7ae526e661"} err="failed to get container status \"82adce2724a2f1c30bd318873c36ef2e4146fcce1295deaf1dffee7ae526e661\": rpc error: code = NotFound desc = could not find container \"82adce2724a2f1c30bd318873c36ef2e4146fcce1295deaf1dffee7ae526e661\": container with ID starting with 82adce2724a2f1c30bd318873c36ef2e4146fcce1295deaf1dffee7ae526e661 not found: ID does not exist"
Jul 30 19:31:47.682066 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.682046    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-wmfxj\""
Jul 30 19:31:47.682066 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.682059    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 30 19:31:47.688125 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.688101    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-d9nzc"]
Jul 30 19:31:47.702719 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.702684    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-869fdf5f7b-h9x47"]
Jul 30 19:31:47.710271 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.710245    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-869fdf5f7b-h9x47"]
Jul 30 19:31:47.766901 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.766860    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-759r6\" (UniqueName: \"kubernetes.io/projected/ace7990c-649f-496b-8cd1-52ca9a626f9f-kube-api-access-759r6\") pod \"dns-operator-controller-manager-864dd78c4c-d9nzc\" (UID: \"ace7990c-649f-496b-8cd1-52ca9a626f9f\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d9nzc"
Jul 30 19:31:47.868333 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.868292    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-759r6\" (UniqueName: \"kubernetes.io/projected/ace7990c-649f-496b-8cd1-52ca9a626f9f-kube-api-access-759r6\") pod \"dns-operator-controller-manager-864dd78c4c-d9nzc\" (UID: \"ace7990c-649f-496b-8cd1-52ca9a626f9f\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d9nzc"
Jul 30 19:31:47.888660 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.888634    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-759r6\" (UniqueName: \"kubernetes.io/projected/ace7990c-649f-496b-8cd1-52ca9a626f9f-kube-api-access-759r6\") pod \"dns-operator-controller-manager-864dd78c4c-d9nzc\" (UID: \"ace7990c-649f-496b-8cd1-52ca9a626f9f\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d9nzc"
Jul 30 19:31:47.990394 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:47.990355    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d9nzc"
Jul 30 19:31:48.130663 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:48.130638    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-d9nzc"]
Jul 30 19:31:48.132736 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:31:48.132710    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podace7990c_649f_496b_8cd1_52ca9a626f9f.slice/crio-e9c12e0b90f7a53c87a02a86323a04f03408d1ce63364ba381f72a213c7bae04 WatchSource:0}: Error finding container e9c12e0b90f7a53c87a02a86323a04f03408d1ce63364ba381f72a213c7bae04: Status 404 returned error can't find the container with id e9c12e0b90f7a53c87a02a86323a04f03408d1ce63364ba381f72a213c7bae04
Jul 30 19:31:48.486232 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:48.486192    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="539b3bf5-a852-4029-8107-2d89ee7d6e5b" path="/var/lib/kubelet/pods/539b3bf5-a852-4029-8107-2d89ee7d6e5b/volumes"
Jul 30 19:31:48.677178 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:48.677138    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d9nzc" event={"ID":"ace7990c-649f-496b-8cd1-52ca9a626f9f","Type":"ContainerStarted","Data":"e9c12e0b90f7a53c87a02a86323a04f03408d1ce63364ba381f72a213c7bae04"}
Jul 30 19:31:49.232512 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:49.232481    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:31:50.690025 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:50.689987    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d9nzc" event={"ID":"ace7990c-649f-496b-8cd1-52ca9a626f9f","Type":"ContainerStarted","Data":"fc40409a49d94145d09098a48abd02bb8c3985cfb9e08624b155c8b06903bf30"}
Jul 30 19:31:50.690488 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:50.690075    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d9nzc"
Jul 30 19:31:50.708213 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:50.708161    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d9nzc" podStartSLOduration=1.358463292 podStartE2EDuration="3.708144941s" totalImagesPullingTime="2.349681649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:31:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:31:50.706818974 +0000 UTC m=+438.671400057" watchObservedRunningTime="2026-07-30 19:31:50.708144941 +0000 UTC m=+438.672726094"
Jul 30 19:31:51.155423 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:51.155385    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j69sw"]
Jul 30 19:31:51.158899 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:51.158881    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j69sw"
Jul 30 19:31:51.161444 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:51.161421    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-8lznw\""
Jul 30 19:31:51.175686 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:51.175660    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j69sw"]
Jul 30 19:31:51.300749 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:51.300707    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kn8g\" (UniqueName: \"kubernetes.io/projected/f4345b99-01dc-41f9-a8fa-917b8cbab444-kube-api-access-9kn8g\") pod \"limitador-operator-controller-manager-859d7b78bd-j69sw\" (UID: \"f4345b99-01dc-41f9-a8fa-917b8cbab444\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j69sw"
Jul 30 19:31:51.402015 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:51.401966    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kn8g\" (UniqueName: \"kubernetes.io/projected/f4345b99-01dc-41f9-a8fa-917b8cbab444-kube-api-access-9kn8g\") pod \"limitador-operator-controller-manager-859d7b78bd-j69sw\" (UID: \"f4345b99-01dc-41f9-a8fa-917b8cbab444\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j69sw"
Jul 30 19:31:51.412532 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:51.412455    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kn8g\" (UniqueName: \"kubernetes.io/projected/f4345b99-01dc-41f9-a8fa-917b8cbab444-kube-api-access-9kn8g\") pod \"limitador-operator-controller-manager-859d7b78bd-j69sw\" (UID: \"f4345b99-01dc-41f9-a8fa-917b8cbab444\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j69sw"
Jul 30 19:31:51.479092 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:51.479054    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j69sw"
Jul 30 19:31:51.626122 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:51.626097    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j69sw"]
Jul 30 19:31:51.627720 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:31:51.627689    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4345b99_01dc_41f9_a8fa_917b8cbab444.slice/crio-565c31c43339e42b8a7eac9d1a19865bc40988ecdf4df00c0c3adc94be09739f WatchSource:0}: Error finding container 565c31c43339e42b8a7eac9d1a19865bc40988ecdf4df00c0c3adc94be09739f: Status 404 returned error can't find the container with id 565c31c43339e42b8a7eac9d1a19865bc40988ecdf4df00c0c3adc94be09739f
Jul 30 19:31:51.668019 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:51.667916    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:31:51.696222 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:51.696186    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j69sw" event={"ID":"f4345b99-01dc-41f9-a8fa-917b8cbab444","Type":"ContainerStarted","Data":"565c31c43339e42b8a7eac9d1a19865bc40988ecdf4df00c0c3adc94be09739f"}
Jul 30 19:31:51.831977 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:51.831923    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:31:53.082772 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:53.082738    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:31:54.714840 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:54.714809    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j69sw" event={"ID":"f4345b99-01dc-41f9-a8fa-917b8cbab444","Type":"ContainerStarted","Data":"d906d409e9f766f5a7f071a2ac5fd9b7ebf7135454a139998fe5da8702ea1ad7"}
Jul 30 19:31:54.715234 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:54.714941    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j69sw"
Jul 30 19:31:54.731552 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:54.731401    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j69sw" podStartSLOduration=1.19614935 podStartE2EDuration="3.731384144s" totalImagesPullingTime="2.535234794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:31:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:31:54.730788108 +0000 UTC m=+442.695369239" watchObservedRunningTime="2026-07-30 19:31:54.731384144 +0000 UTC m=+442.695965224"
Jul 30 19:31:55.823771 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:55.823736    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95t4q"]
Jul 30 19:31:55.827023 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:55.827005    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95t4q"
Jul 30 19:31:55.829701 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:55.829677    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mpd9r\""
Jul 30 19:31:55.838974 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:55.838936    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95t4q"]
Jul 30 19:31:55.945062 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:55.945013    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4m9j\" (UniqueName: \"kubernetes.io/projected/a45d74f0-6f06-42d0-a2b3-9840a3c5890c-kube-api-access-z4m9j\") pod \"kuadrant-operator-controller-manager-8684c9ff49-95t4q\" (UID: \"a45d74f0-6f06-42d0-a2b3-9840a3c5890c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95t4q"
Jul 30 19:31:55.945062 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:55.945075    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a45d74f0-6f06-42d0-a2b3-9840a3c5890c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-95t4q\" (UID: \"a45d74f0-6f06-42d0-a2b3-9840a3c5890c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95t4q"
Jul 30 19:31:56.046012 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:56.045948    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4m9j\" (UniqueName: \"kubernetes.io/projected/a45d74f0-6f06-42d0-a2b3-9840a3c5890c-kube-api-access-z4m9j\") pod \"kuadrant-operator-controller-manager-8684c9ff49-95t4q\" (UID: \"a45d74f0-6f06-42d0-a2b3-9840a3c5890c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95t4q"
Jul 30 19:31:56.046149 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:56.046023    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a45d74f0-6f06-42d0-a2b3-9840a3c5890c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-95t4q\" (UID: \"a45d74f0-6f06-42d0-a2b3-9840a3c5890c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95t4q"
Jul 30 19:31:56.046381 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:56.046364    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a45d74f0-6f06-42d0-a2b3-9840a3c5890c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-95t4q\" (UID: \"a45d74f0-6f06-42d0-a2b3-9840a3c5890c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95t4q"
Jul 30 19:31:56.053987 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:56.053937    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4m9j\" (UniqueName: \"kubernetes.io/projected/a45d74f0-6f06-42d0-a2b3-9840a3c5890c-kube-api-access-z4m9j\") pod \"kuadrant-operator-controller-manager-8684c9ff49-95t4q\" (UID: \"a45d74f0-6f06-42d0-a2b3-9840a3c5890c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95t4q"
Jul 30 19:31:56.137253 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:56.137146    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95t4q"
Jul 30 19:31:56.225766 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:56.225737    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:31:56.297357 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:56.297332    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95t4q"]
Jul 30 19:31:56.299800 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:31:56.299767    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda45d74f0_6f06_42d0_a2b3_9840a3c5890c.slice/crio-a580a8d2d613cfa2ecbd949b1df6d196d7e2f4b5943c4d1521f7840586ab9b50 WatchSource:0}: Error finding container a580a8d2d613cfa2ecbd949b1df6d196d7e2f4b5943c4d1521f7840586ab9b50: Status 404 returned error can't find the container with id a580a8d2d613cfa2ecbd949b1df6d196d7e2f4b5943c4d1521f7840586ab9b50
Jul 30 19:31:56.726910 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:56.726874    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95t4q" event={"ID":"a45d74f0-6f06-42d0-a2b3-9840a3c5890c","Type":"ContainerStarted","Data":"a580a8d2d613cfa2ecbd949b1df6d196d7e2f4b5943c4d1521f7840586ab9b50"}
Jul 30 19:31:57.114139 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:57.114047    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:31:57.216455 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:57.216422    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:31:59.668592 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:31:59.668552    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:01.038377 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:01.038339    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:01.138326 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:01.138254    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:01.699063 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:01.698944    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d9nzc"
Jul 30 19:32:01.760474 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:01.760437    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95t4q" event={"ID":"a45d74f0-6f06-42d0-a2b3-9840a3c5890c","Type":"ContainerStarted","Data":"3a6ee5ae35e6d230288d4a2a2a02421fd778203cf69d25a81e1e8d3da940c39a"}
Jul 30 19:32:01.760622 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:01.760566    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95t4q"
Jul 30 19:32:01.779274 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:01.779220    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95t4q" podStartSLOduration=1.658386855 podStartE2EDuration="6.77919954s" totalImagesPullingTime="5.120812685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:31:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:32:01.776809307 +0000 UTC m=+449.741390399" watchObservedRunningTime="2026-07-30 19:32:01.77919954 +0000 UTC m=+449.743780621"
Jul 30 19:32:04.048600 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:04.048565    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-sv8wn"]
Jul 30 19:32:04.052502 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:04.052474    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sv8wn"
Jul 30 19:32:04.054629 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:04.054607    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 30 19:32:04.054773 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:04.054757    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gnqwl\""
Jul 30 19:32:04.054834 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:04.054818    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 30 19:32:04.060051 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:04.060015    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:04.062381 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:04.062356    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-sv8wn"]
Jul 30 19:32:04.233757 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:04.233720    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl9zs\" (UniqueName: \"kubernetes.io/projected/502f8628-f1d0-4b2d-908a-8b09e2211e59-kube-api-access-jl9zs\") pod \"kuadrant-console-plugin-66b5774985-sv8wn\" (UID: \"502f8628-f1d0-4b2d-908a-8b09e2211e59\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sv8wn"
Jul 30 19:32:04.233936 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:04.233774    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/502f8628-f1d0-4b2d-908a-8b09e2211e59-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-sv8wn\" (UID: \"502f8628-f1d0-4b2d-908a-8b09e2211e59\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sv8wn"
Jul 30 19:32:04.233936 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:04.233846    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/502f8628-f1d0-4b2d-908a-8b09e2211e59-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-sv8wn\" (UID: \"502f8628-f1d0-4b2d-908a-8b09e2211e59\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sv8wn"
Jul 30 19:32:04.334580 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:04.334467    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl9zs\" (UniqueName: \"kubernetes.io/projected/502f8628-f1d0-4b2d-908a-8b09e2211e59-kube-api-access-jl9zs\") pod \"kuadrant-console-plugin-66b5774985-sv8wn\" (UID: \"502f8628-f1d0-4b2d-908a-8b09e2211e59\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sv8wn"
Jul 30 19:32:04.334580 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:04.334533    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/502f8628-f1d0-4b2d-908a-8b09e2211e59-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-sv8wn\" (UID: \"502f8628-f1d0-4b2d-908a-8b09e2211e59\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sv8wn"
Jul 30 19:32:04.334801 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:04.334594    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/502f8628-f1d0-4b2d-908a-8b09e2211e59-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-sv8wn\" (UID: \"502f8628-f1d0-4b2d-908a-8b09e2211e59\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sv8wn"
Jul 30 19:32:04.335370 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:04.335348    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/502f8628-f1d0-4b2d-908a-8b09e2211e59-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-sv8wn\" (UID: \"502f8628-f1d0-4b2d-908a-8b09e2211e59\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sv8wn"
Jul 30 19:32:04.337205 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:04.337186    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/502f8628-f1d0-4b2d-908a-8b09e2211e59-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-sv8wn\" (UID: \"502f8628-f1d0-4b2d-908a-8b09e2211e59\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sv8wn"
Jul 30 19:32:04.342456 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:04.342430    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl9zs\" (UniqueName: \"kubernetes.io/projected/502f8628-f1d0-4b2d-908a-8b09e2211e59-kube-api-access-jl9zs\") pod \"kuadrant-console-plugin-66b5774985-sv8wn\" (UID: \"502f8628-f1d0-4b2d-908a-8b09e2211e59\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sv8wn"
Jul 30 19:32:04.362889 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:04.362850    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sv8wn"
Jul 30 19:32:04.500038 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:04.499948    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-sv8wn"]
Jul 30 19:32:04.502503 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:32:04.502472    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod502f8628_f1d0_4b2d_908a_8b09e2211e59.slice/crio-200cb2902a4f12459bd4a5aee079c3b38ccbbc00d1e5d78ffcb23b056fb6e1e8 WatchSource:0}: Error finding container 200cb2902a4f12459bd4a5aee079c3b38ccbbc00d1e5d78ffcb23b056fb6e1e8: Status 404 returned error can't find the container with id 200cb2902a4f12459bd4a5aee079c3b38ccbbc00d1e5d78ffcb23b056fb6e1e8
Jul 30 19:32:04.718648 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:04.718615    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:04.778397 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:04.778361    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sv8wn" event={"ID":"502f8628-f1d0-4b2d-908a-8b09e2211e59","Type":"ContainerStarted","Data":"200cb2902a4f12459bd4a5aee079c3b38ccbbc00d1e5d78ffcb23b056fb6e1e8"}
Jul 30 19:32:04.837991 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:04.837939    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:04.938535 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:04.938494    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:05.723391 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:05.723362    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j69sw"
Jul 30 19:32:08.237831 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:08.237788    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:08.286137 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:08.286100    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:12.768283 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:12.768246    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95t4q"
Jul 30 19:32:13.827642 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:13.827598    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-h8s4h"]
Jul 30 19:32:13.848756 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:13.848722    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-h8s4h"
Jul 30 19:32:13.854801 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:13.854755    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-h8s4h"]
Jul 30 19:32:13.918968 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:13.918915    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33a96823-5ef7-43a7-8684-1d6aa5bc1a9d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-h8s4h\" (UID: \"33a96823-5ef7-43a7-8684-1d6aa5bc1a9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-h8s4h"
Jul 30 19:32:13.919145 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:13.919015    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq5ln\" (UniqueName: \"kubernetes.io/projected/33a96823-5ef7-43a7-8684-1d6aa5bc1a9d-kube-api-access-zq5ln\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-h8s4h\" (UID: \"33a96823-5ef7-43a7-8684-1d6aa5bc1a9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-h8s4h"
Jul 30 19:32:14.020392 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.020350    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33a96823-5ef7-43a7-8684-1d6aa5bc1a9d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-h8s4h\" (UID: \"33a96823-5ef7-43a7-8684-1d6aa5bc1a9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-h8s4h"
Jul 30 19:32:14.020590 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.020405    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq5ln\" (UniqueName: \"kubernetes.io/projected/33a96823-5ef7-43a7-8684-1d6aa5bc1a9d-kube-api-access-zq5ln\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-h8s4h\" (UID: \"33a96823-5ef7-43a7-8684-1d6aa5bc1a9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-h8s4h"
Jul 30 19:32:14.020826 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.020799    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33a96823-5ef7-43a7-8684-1d6aa5bc1a9d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-h8s4h\" (UID: \"33a96823-5ef7-43a7-8684-1d6aa5bc1a9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-h8s4h"
Jul 30 19:32:14.033092 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.033054    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq5ln\" (UniqueName: \"kubernetes.io/projected/33a96823-5ef7-43a7-8684-1d6aa5bc1a9d-kube-api-access-zq5ln\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-h8s4h\" (UID: \"33a96823-5ef7-43a7-8684-1d6aa5bc1a9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-h8s4h"
Jul 30 19:32:14.164979 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.164869    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-h8s4h"
Jul 30 19:32:14.452622 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.452526    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-h8s4h"]
Jul 30 19:32:14.460160 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.460102    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-h8s4h"]
Jul 30 19:32:14.472076 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.471976    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95t4q"]
Jul 30 19:32:14.472411 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.472200    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95t4q" podUID="a45d74f0-6f06-42d0-a2b3-9840a3c5890c" containerName="manager" containerID="cri-o://3a6ee5ae35e6d230288d4a2a2a02421fd778203cf69d25a81e1e8d3da940c39a" gracePeriod=2
Jul 30 19:32:14.476865 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.476745    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86prw"]
Jul 30 19:32:14.480126 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:32:14.480090    2554 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"a45d74f0-6f06-42d0-a2b3-9840a3c5890c\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-95t4q\": pods \"kuadrant-operator-controller-manager-8684c9ff49-95t4q\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95t4q"
Jul 30 19:32:14.493144 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.493106    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95t4q"]
Jul 30 19:32:14.496209 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.494322    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86prw"
Jul 30 19:32:14.497831 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:32:14.497638    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-95t4q\" is forbidden: User \"system:node:ip-10-0-134-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-180.ec2.internal' and this object" podUID="a45d74f0-6f06-42d0-a2b3-9840a3c5890c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95t4q"
Jul 30 19:32:14.506461 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.506434    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86prw"]
Jul 30 19:32:14.506658 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.506502    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j69sw"]
Jul 30 19:32:14.506658 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.506539    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wwhv"]
Jul 30 19:32:14.507129 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.507106    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a45d74f0-6f06-42d0-a2b3-9840a3c5890c" containerName="manager"
Jul 30 19:32:14.507129 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.507129    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="a45d74f0-6f06-42d0-a2b3-9840a3c5890c" containerName="manager"
Jul 30 19:32:14.507308 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.507237    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="a45d74f0-6f06-42d0-a2b3-9840a3c5890c" containerName="manager"
Jul 30 19:32:14.507422 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.507390    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j69sw" podUID="f4345b99-01dc-41f9-a8fa-917b8cbab444" containerName="manager" containerID="cri-o://d906d409e9f766f5a7f071a2ac5fd9b7ebf7135454a139998fe5da8702ea1ad7" gracePeriod=2
Jul 30 19:32:14.510656 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.510628    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j69sw"]
Jul 30 19:32:14.510794 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.510774    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wwhv"
Jul 30 19:32:14.528455 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.527815    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4fbfe24c-0bfe-408e-9be5-03ca2a14acec-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4wwhv\" (UID: \"4fbfe24c-0bfe-408e-9be5-03ca2a14acec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wwhv"
Jul 30 19:32:14.528455 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.527881    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwbqq\" (UniqueName: \"kubernetes.io/projected/b6a6f617-7c9d-497d-b347-7ea4c0dd3b24-kube-api-access-bwbqq\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-86prw\" (UID: \"b6a6f617-7c9d-497d-b347-7ea4c0dd3b24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86prw"
Jul 30 19:32:14.528455 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.528068    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b6a6f617-7c9d-497d-b347-7ea4c0dd3b24-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-86prw\" (UID: \"b6a6f617-7c9d-497d-b347-7ea4c0dd3b24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86prw"
Jul 30 19:32:14.528455 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.528105    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6nps\" (UniqueName: \"kubernetes.io/projected/4fbfe24c-0bfe-408e-9be5-03ca2a14acec-kube-api-access-j6nps\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4wwhv\" (UID: \"4fbfe24c-0bfe-408e-9be5-03ca2a14acec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wwhv"
Jul 30 19:32:14.530750 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:32:14.530249    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j69sw\" is forbidden: User \"system:node:ip-10-0-134-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-180.ec2.internal' and this object" podUID="f4345b99-01dc-41f9-a8fa-917b8cbab444" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j69sw"
Jul 30 19:32:14.531059 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.530982    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wwhv"]
Jul 30 19:32:14.629064 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.629022    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4fbfe24c-0bfe-408e-9be5-03ca2a14acec-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4wwhv\" (UID: \"4fbfe24c-0bfe-408e-9be5-03ca2a14acec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wwhv"
Jul 30 19:32:14.629237 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.629087    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwbqq\" (UniqueName: \"kubernetes.io/projected/b6a6f617-7c9d-497d-b347-7ea4c0dd3b24-kube-api-access-bwbqq\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-86prw\" (UID: \"b6a6f617-7c9d-497d-b347-7ea4c0dd3b24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86prw"
Jul 30 19:32:14.629237 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.629169    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b6a6f617-7c9d-497d-b347-7ea4c0dd3b24-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-86prw\" (UID: \"b6a6f617-7c9d-497d-b347-7ea4c0dd3b24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86prw"
Jul 30 19:32:14.629237 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.629196    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6nps\" (UniqueName: \"kubernetes.io/projected/4fbfe24c-0bfe-408e-9be5-03ca2a14acec-kube-api-access-j6nps\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4wwhv\" (UID: \"4fbfe24c-0bfe-408e-9be5-03ca2a14acec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wwhv"
Jul 30 19:32:14.629423 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.629406    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4fbfe24c-0bfe-408e-9be5-03ca2a14acec-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4wwhv\" (UID: \"4fbfe24c-0bfe-408e-9be5-03ca2a14acec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wwhv"
Jul 30 19:32:14.629652 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.629628    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b6a6f617-7c9d-497d-b347-7ea4c0dd3b24-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-86prw\" (UID: \"b6a6f617-7c9d-497d-b347-7ea4c0dd3b24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86prw"
Jul 30 19:32:14.639589 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.639561    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6nps\" (UniqueName: \"kubernetes.io/projected/4fbfe24c-0bfe-408e-9be5-03ca2a14acec-kube-api-access-j6nps\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4wwhv\" (UID: \"4fbfe24c-0bfe-408e-9be5-03ca2a14acec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wwhv"
Jul 30 19:32:14.639748 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.639631    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwbqq\" (UniqueName: \"kubernetes.io/projected/b6a6f617-7c9d-497d-b347-7ea4c0dd3b24-kube-api-access-bwbqq\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-86prw\" (UID: \"b6a6f617-7c9d-497d-b347-7ea4c0dd3b24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86prw"
Jul 30 19:32:14.898290 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.898245    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86prw"
Jul 30 19:32:14.909331 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:14.909301    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wwhv"
Jul 30 19:32:16.721532 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:16.721485    2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j69sw" podUID="f4345b99-01dc-41f9-a8fa-917b8cbab444" containerName="manager" probeResult="failure" output="Get \"http://10.132.0.42:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 30 19:32:17.244996 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:17.244935    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:17.302497 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:17.302456    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:17.847895 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:17.847857    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:17.948651 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:17.948617    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:23.646708 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:23.646664    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:24.126353 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:24.126316    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:26.935047 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:26.934986    2554 generic.go:356] "Generic (PLEG): container finished" podID="f4345b99-01dc-41f9-a8fa-917b8cbab444" containerID="d906d409e9f766f5a7f071a2ac5fd9b7ebf7135454a139998fe5da8702ea1ad7" exitCode=0
Jul 30 19:32:26.940458 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:26.939984    2554 generic.go:356] "Generic (PLEG): container finished" podID="a45d74f0-6f06-42d0-a2b3-9840a3c5890c" containerID="3a6ee5ae35e6d230288d4a2a2a02421fd778203cf69d25a81e1e8d3da940c39a" exitCode=0
Jul 30 19:32:27.189939 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:27.189903    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95t4q"
Jul 30 19:32:27.192284 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:32:27.192246    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-95t4q\" is forbidden: User \"system:node:ip-10-0-134-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-180.ec2.internal' and this object" podUID="a45d74f0-6f06-42d0-a2b3-9840a3c5890c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95t4q"
Jul 30 19:32:27.211007 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:27.210907    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86prw"]
Jul 30 19:32:27.212280 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:32:27.212244    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6a6f617_7c9d_497d_b347_7ea4c0dd3b24.slice/crio-b08a0ff1b241715c6b0ed0b76f2605b75e8fadb9366f9e74cc169a86f36319be WatchSource:0}: Error finding container b08a0ff1b241715c6b0ed0b76f2605b75e8fadb9366f9e74cc169a86f36319be: Status 404 returned error can't find the container with id b08a0ff1b241715c6b0ed0b76f2605b75e8fadb9366f9e74cc169a86f36319be
Jul 30 19:32:27.212684 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:27.212658    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j69sw"
Jul 30 19:32:27.212684 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:32:27.212670    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fbfe24c_0bfe_408e_9be5_03ca2a14acec.slice/crio-537055ac6beebe310a12a747df1c7c49580ec0b4fd5008c0d9b969932e70adc3 WatchSource:0}: Error finding container 537055ac6beebe310a12a747df1c7c49580ec0b4fd5008c0d9b969932e70adc3: Status 404 returned error can't find the container with id 537055ac6beebe310a12a747df1c7c49580ec0b4fd5008c0d9b969932e70adc3
Jul 30 19:32:27.213577 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:27.213529    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wwhv"]
Jul 30 19:32:27.214744 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:32:27.214714    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j69sw\" is forbidden: User \"system:node:ip-10-0-134-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-180.ec2.internal' and this object" podUID="f4345b99-01dc-41f9-a8fa-917b8cbab444" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j69sw"
Jul 30 19:32:27.216520 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:32:27.216497    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-95t4q\" is forbidden: User \"system:node:ip-10-0-134-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-180.ec2.internal' and this object" podUID="a45d74f0-6f06-42d0-a2b3-9840a3c5890c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95t4q"
Jul 30 19:32:27.263971 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:27.263932    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9kn8g\" (UniqueName: \"kubernetes.io/projected/f4345b99-01dc-41f9-a8fa-917b8cbab444-kube-api-access-9kn8g\") pod \"f4345b99-01dc-41f9-a8fa-917b8cbab444\" (UID: \"f4345b99-01dc-41f9-a8fa-917b8cbab444\") "
Jul 30 19:32:27.264088 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:27.264050    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4m9j\" (UniqueName: \"kubernetes.io/projected/a45d74f0-6f06-42d0-a2b3-9840a3c5890c-kube-api-access-z4m9j\") pod \"a45d74f0-6f06-42d0-a2b3-9840a3c5890c\" (UID: \"a45d74f0-6f06-42d0-a2b3-9840a3c5890c\") "
Jul 30 19:32:27.264088 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:27.264077    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a45d74f0-6f06-42d0-a2b3-9840a3c5890c-extensions-socket-volume\") pod \"a45d74f0-6f06-42d0-a2b3-9840a3c5890c\" (UID: \"a45d74f0-6f06-42d0-a2b3-9840a3c5890c\") "
Jul 30 19:32:27.264578 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:27.264543    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a45d74f0-6f06-42d0-a2b3-9840a3c5890c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a45d74f0-6f06-42d0-a2b3-9840a3c5890c" (UID: "a45d74f0-6f06-42d0-a2b3-9840a3c5890c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:32:27.265912 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:27.265884    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4345b99-01dc-41f9-a8fa-917b8cbab444-kube-api-access-9kn8g" (OuterVolumeSpecName: "kube-api-access-9kn8g") pod "f4345b99-01dc-41f9-a8fa-917b8cbab444" (UID: "f4345b99-01dc-41f9-a8fa-917b8cbab444"). InnerVolumeSpecName "kube-api-access-9kn8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:32:27.266028 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:27.265994    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a45d74f0-6f06-42d0-a2b3-9840a3c5890c-kube-api-access-z4m9j" (OuterVolumeSpecName: "kube-api-access-z4m9j") pod "a45d74f0-6f06-42d0-a2b3-9840a3c5890c" (UID: "a45d74f0-6f06-42d0-a2b3-9840a3c5890c"). InnerVolumeSpecName "kube-api-access-z4m9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:32:27.364983 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:27.364942    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4m9j\" (UniqueName: \"kubernetes.io/projected/a45d74f0-6f06-42d0-a2b3-9840a3c5890c-kube-api-access-z4m9j\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:32:27.365097 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:27.364986    2554 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a45d74f0-6f06-42d0-a2b3-9840a3c5890c-extensions-socket-volume\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:32:27.365097 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:27.364998    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9kn8g\" (UniqueName: \"kubernetes.io/projected/f4345b99-01dc-41f9-a8fa-917b8cbab444-kube-api-access-9kn8g\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:32:27.837626 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:27.837589    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:27.939370 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:27.939338    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:27.953135 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:27.953083    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sv8wn" event={"ID":"502f8628-f1d0-4b2d-908a-8b09e2211e59","Type":"ContainerStarted","Data":"4a64976ab53b54de9fd2e136a698463b58b94b3c229fd683da84f04540104d40"}
Jul 30 19:32:27.954877 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:27.954846    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86prw" event={"ID":"b6a6f617-7c9d-497d-b347-7ea4c0dd3b24","Type":"ContainerStarted","Data":"644e17ba5bdc7089fd45560d19c56a0525878c70ad2bdb92f3bce8c7de976f7b"}
Jul 30 19:32:27.955065 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:27.954932    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86prw"
Jul 30 19:32:27.955065 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:27.954971    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86prw" event={"ID":"b6a6f617-7c9d-497d-b347-7ea4c0dd3b24","Type":"ContainerStarted","Data":"b08a0ff1b241715c6b0ed0b76f2605b75e8fadb9366f9e74cc169a86f36319be"}
Jul 30 19:32:27.955279 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:32:27.955252    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-95t4q\" is forbidden: User \"system:node:ip-10-0-134-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-180.ec2.internal' and this object" podUID="a45d74f0-6f06-42d0-a2b3-9840a3c5890c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95t4q"
Jul 30 19:32:27.956254 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:27.956237    2554 scope.go:117] "RemoveContainer" containerID="d906d409e9f766f5a7f071a2ac5fd9b7ebf7135454a139998fe5da8702ea1ad7"
Jul 30 19:32:27.956254 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:27.956242    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j69sw"
Jul 30 19:32:27.956976 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:32:27.956935    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j69sw\" is forbidden: User \"system:node:ip-10-0-134-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-180.ec2.internal' and this object" podUID="f4345b99-01dc-41f9-a8fa-917b8cbab444" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j69sw"
Jul 30 19:32:27.957874 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:27.957858    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95t4q"
Jul 30 19:32:27.959964 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:27.959925    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wwhv" event={"ID":"4fbfe24c-0bfe-408e-9be5-03ca2a14acec","Type":"ContainerStarted","Data":"64d2644f084b14ef6ea44aff178b45b2a0d558ee02ce5a4b95cc8bdda34f5ba6"}
Jul 30 19:32:27.960073 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:27.959991    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wwhv" event={"ID":"4fbfe24c-0bfe-408e-9be5-03ca2a14acec","Type":"ContainerStarted","Data":"537055ac6beebe310a12a747df1c7c49580ec0b4fd5008c0d9b969932e70adc3"}
Jul 30 19:32:27.960073 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:27.960043    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wwhv"
Jul 30 19:32:27.966786 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:27.966767    2554 scope.go:117] "RemoveContainer" containerID="3a6ee5ae35e6d230288d4a2a2a02421fd778203cf69d25a81e1e8d3da940c39a"
Jul 30 19:32:27.972757 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:27.972714    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sv8wn" podStartSLOduration=1.263505355 podStartE2EDuration="23.972698252s" totalImagesPullingTime="22.709192897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:32:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:32:27.971180008 +0000 UTC m=+475.935761082" watchObservedRunningTime="2026-07-30 19:32:27.972698252 +0000 UTC m=+475.937279333"
Jul 30 19:32:27.973127 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:32:27.973097    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-j69sw\" is forbidden: User \"system:node:ip-10-0-134-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-180.ec2.internal' and this object" podUID="f4345b99-01dc-41f9-a8fa-917b8cbab444" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j69sw"
Jul 30 19:32:27.974998 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:32:27.974971    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-95t4q\" is forbidden: User \"system:node:ip-10-0-134-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-180.ec2.internal' and this object" podUID="a45d74f0-6f06-42d0-a2b3-9840a3c5890c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-95t4q"
Jul 30 19:32:27.996933 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:27.996878    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wwhv" podStartSLOduration=13.996859789 podStartE2EDuration="13.996859789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:32:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:32:27.995478221 +0000 UTC m=+475.960059303" watchObservedRunningTime="2026-07-30 19:32:27.996859789 +0000 UTC m=+475.961440872"
Jul 30 19:32:28.013858 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:28.013399    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86prw" podStartSLOduration=14.013377288 podStartE2EDuration="14.013377288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:32:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:32:28.011660117 +0000 UTC m=+475.976241199" watchObservedRunningTime="2026-07-30 19:32:28.013377288 +0000 UTC m=+475.977958370"
Jul 30 19:32:28.486977 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:28.486922    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a45d74f0-6f06-42d0-a2b3-9840a3c5890c" path="/var/lib/kubelet/pods/a45d74f0-6f06-42d0-a2b3-9840a3c5890c/volumes"
Jul 30 19:32:28.487299 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:28.487285    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4345b99-01dc-41f9-a8fa-917b8cbab444" path="/var/lib/kubelet/pods/f4345b99-01dc-41f9-a8fa-917b8cbab444/volumes"
Jul 30 19:32:29.330827 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:32:29.330782    2554 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-h8s4h_kuadrant-system_33a96823-5ef7-43a7-8684-1d6aa5bc1a9d_0(bf60bd8d41623da183da481aa33786404c97fef6de1971bf71a44b9c508dbbcd): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-h8s4h to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}"
Jul 30 19:32:29.331332 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:32:29.330851    2554 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-h8s4h_kuadrant-system_33a96823-5ef7-43a7-8684-1d6aa5bc1a9d_0(bf60bd8d41623da183da481aa33786404c97fef6de1971bf71a44b9c508dbbcd): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-h8s4h to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-h8s4h"
Jul 30 19:32:29.973391 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:29.973358    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-h8s4h"
Jul 30 19:32:29.976062 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:32:29.976035    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-h8s4h\" is forbidden: User \"system:node:ip-10-0-134-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-180.ec2.internal' and this object" podUID="33a96823-5ef7-43a7-8684-1d6aa5bc1a9d" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-h8s4h"
Jul 30 19:32:30.076102 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:30.076063    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-h8s4h"
Jul 30 19:32:30.078291 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:32:30.078265    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-h8s4h\" is forbidden: User \"system:node:ip-10-0-134-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-180.ec2.internal' and this object" podUID="33a96823-5ef7-43a7-8684-1d6aa5bc1a9d" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-h8s4h"
Jul 30 19:32:30.191702 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:30.191670    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zq5ln\" (UniqueName: \"kubernetes.io/projected/33a96823-5ef7-43a7-8684-1d6aa5bc1a9d-kube-api-access-zq5ln\") pod \"33a96823-5ef7-43a7-8684-1d6aa5bc1a9d\" (UID: \"33a96823-5ef7-43a7-8684-1d6aa5bc1a9d\") "
Jul 30 19:32:30.191884 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:30.191765    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33a96823-5ef7-43a7-8684-1d6aa5bc1a9d-extensions-socket-volume\") pod \"33a96823-5ef7-43a7-8684-1d6aa5bc1a9d\" (UID: \"33a96823-5ef7-43a7-8684-1d6aa5bc1a9d\") "
Jul 30 19:32:30.192070 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:30.192043    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33a96823-5ef7-43a7-8684-1d6aa5bc1a9d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "33a96823-5ef7-43a7-8684-1d6aa5bc1a9d" (UID: "33a96823-5ef7-43a7-8684-1d6aa5bc1a9d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:32:30.193817 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:30.193788    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33a96823-5ef7-43a7-8684-1d6aa5bc1a9d-kube-api-access-zq5ln" (OuterVolumeSpecName: "kube-api-access-zq5ln") pod "33a96823-5ef7-43a7-8684-1d6aa5bc1a9d" (UID: "33a96823-5ef7-43a7-8684-1d6aa5bc1a9d"). InnerVolumeSpecName "kube-api-access-zq5ln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:32:30.292455 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:30.292368    2554 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33a96823-5ef7-43a7-8684-1d6aa5bc1a9d-extensions-socket-volume\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:32:30.292455 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:30.292399    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zq5ln\" (UniqueName: \"kubernetes.io/projected/33a96823-5ef7-43a7-8684-1d6aa5bc1a9d-kube-api-access-zq5ln\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:32:30.487724 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:30.487687    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33a96823-5ef7-43a7-8684-1d6aa5bc1a9d" path="/var/lib/kubelet/pods/33a96823-5ef7-43a7-8684-1d6aa5bc1a9d/volumes"
Jul 30 19:32:30.978413 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:30.978385    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-h8s4h"
Jul 30 19:32:30.980866 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:32:30.980830    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-h8s4h\" is forbidden: User \"system:node:ip-10-0-134-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-180.ec2.internal' and this object" podUID="33a96823-5ef7-43a7-8684-1d6aa5bc1a9d" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-h8s4h"
Jul 30 19:32:30.982869 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:32:30.982841    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-h8s4h\" is forbidden: User \"system:node:ip-10-0-134-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-180.ec2.internal' and this object" podUID="33a96823-5ef7-43a7-8684-1d6aa5bc1a9d" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-h8s4h"
Jul 30 19:32:31.238530 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:31.238443    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:31.300974 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:31.300924    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:32.485234 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:32:32.485188    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-h8s4h\" is forbidden: User \"system:node:ip-10-0-134-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-180.ec2.internal' and this object" podUID="33a96823-5ef7-43a7-8684-1d6aa5bc1a9d" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-h8s4h"
Jul 30 19:32:38.970406 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:38.970370    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wwhv"
Jul 30 19:32:38.970938 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:38.970470    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86prw"
Jul 30 19:32:39.038508 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:39.038473    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wwhv"]
Jul 30 19:32:39.038706 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:39.038683    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wwhv" podUID="4fbfe24c-0bfe-408e-9be5-03ca2a14acec" containerName="manager" containerID="cri-o://64d2644f084b14ef6ea44aff178b45b2a0d558ee02ce5a4b95cc8bdda34f5ba6" gracePeriod=10
Jul 30 19:32:39.252920 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:39.252827    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lzkrl"]
Jul 30 19:32:39.253340 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:39.253321    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4345b99-01dc-41f9-a8fa-917b8cbab444" containerName="manager"
Jul 30 19:32:39.253392 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:39.253344    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4345b99-01dc-41f9-a8fa-917b8cbab444" containerName="manager"
Jul 30 19:32:39.253433 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:39.253426    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4345b99-01dc-41f9-a8fa-917b8cbab444" containerName="manager"
Jul 30 19:32:39.255349 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:39.255334    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lzkrl"
Jul 30 19:32:39.269588 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:39.269554    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lzkrl"]
Jul 30 19:32:39.296566 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:39.296534    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wwhv"
Jul 30 19:32:39.372166 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:39.372126    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4fbfe24c-0bfe-408e-9be5-03ca2a14acec-extensions-socket-volume\") pod \"4fbfe24c-0bfe-408e-9be5-03ca2a14acec\" (UID: \"4fbfe24c-0bfe-408e-9be5-03ca2a14acec\") "
Jul 30 19:32:39.372362 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:39.372221    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6nps\" (UniqueName: \"kubernetes.io/projected/4fbfe24c-0bfe-408e-9be5-03ca2a14acec-kube-api-access-j6nps\") pod \"4fbfe24c-0bfe-408e-9be5-03ca2a14acec\" (UID: \"4fbfe24c-0bfe-408e-9be5-03ca2a14acec\") "
Jul 30 19:32:39.372362 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:39.372307    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhg2q\" (UniqueName: \"kubernetes.io/projected/d4b8740b-ed13-4da7-84cc-b60b33f80e54-kube-api-access-mhg2q\") pod \"kuadrant-operator-controller-manager-86878c769f-lzkrl\" (UID: \"d4b8740b-ed13-4da7-84cc-b60b33f80e54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lzkrl"
Jul 30 19:32:39.372362 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:39.372339    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4b8740b-ed13-4da7-84cc-b60b33f80e54-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-lzkrl\" (UID: \"d4b8740b-ed13-4da7-84cc-b60b33f80e54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lzkrl"
Jul 30 19:32:39.372611 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:39.372582    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fbfe24c-0bfe-408e-9be5-03ca2a14acec-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4fbfe24c-0bfe-408e-9be5-03ca2a14acec" (UID: "4fbfe24c-0bfe-408e-9be5-03ca2a14acec"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:32:39.374351 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:39.374328    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fbfe24c-0bfe-408e-9be5-03ca2a14acec-kube-api-access-j6nps" (OuterVolumeSpecName: "kube-api-access-j6nps") pod "4fbfe24c-0bfe-408e-9be5-03ca2a14acec" (UID: "4fbfe24c-0bfe-408e-9be5-03ca2a14acec"). InnerVolumeSpecName "kube-api-access-j6nps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:32:39.473866 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:39.473827    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhg2q\" (UniqueName: \"kubernetes.io/projected/d4b8740b-ed13-4da7-84cc-b60b33f80e54-kube-api-access-mhg2q\") pod \"kuadrant-operator-controller-manager-86878c769f-lzkrl\" (UID: \"d4b8740b-ed13-4da7-84cc-b60b33f80e54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lzkrl"
Jul 30 19:32:39.473866 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:39.473874    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4b8740b-ed13-4da7-84cc-b60b33f80e54-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-lzkrl\" (UID: \"d4b8740b-ed13-4da7-84cc-b60b33f80e54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lzkrl"
Jul 30 19:32:39.474154 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:39.473917    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6nps\" (UniqueName: \"kubernetes.io/projected/4fbfe24c-0bfe-408e-9be5-03ca2a14acec-kube-api-access-j6nps\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:32:39.474154 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:39.473927    2554 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4fbfe24c-0bfe-408e-9be5-03ca2a14acec-extensions-socket-volume\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:32:39.474303 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:39.474281    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4b8740b-ed13-4da7-84cc-b60b33f80e54-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-lzkrl\" (UID: \"d4b8740b-ed13-4da7-84cc-b60b33f80e54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lzkrl"
Jul 30 19:32:39.482429 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:39.482397    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhg2q\" (UniqueName: \"kubernetes.io/projected/d4b8740b-ed13-4da7-84cc-b60b33f80e54-kube-api-access-mhg2q\") pod \"kuadrant-operator-controller-manager-86878c769f-lzkrl\" (UID: \"d4b8740b-ed13-4da7-84cc-b60b33f80e54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lzkrl"
Jul 30 19:32:39.567398 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:39.567302    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lzkrl"
Jul 30 19:32:39.707934 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:39.707901    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lzkrl"]
Jul 30 19:32:39.709803 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:32:39.709773    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4b8740b_ed13_4da7_84cc_b60b33f80e54.slice/crio-3512850fc9492164481bb504b0e779b4fafaeae49cf8f8ecd78db57c81b84e1c WatchSource:0}: Error finding container 3512850fc9492164481bb504b0e779b4fafaeae49cf8f8ecd78db57c81b84e1c: Status 404 returned error can't find the container with id 3512850fc9492164481bb504b0e779b4fafaeae49cf8f8ecd78db57c81b84e1c
Jul 30 19:32:40.029005 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:40.028942    2554 generic.go:356] "Generic (PLEG): container finished" podID="4fbfe24c-0bfe-408e-9be5-03ca2a14acec" containerID="64d2644f084b14ef6ea44aff178b45b2a0d558ee02ce5a4b95cc8bdda34f5ba6" exitCode=0
Jul 30 19:32:40.029005 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:40.029002    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wwhv"
Jul 30 19:32:40.029565 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:40.029051    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wwhv" event={"ID":"4fbfe24c-0bfe-408e-9be5-03ca2a14acec","Type":"ContainerDied","Data":"64d2644f084b14ef6ea44aff178b45b2a0d558ee02ce5a4b95cc8bdda34f5ba6"}
Jul 30 19:32:40.029565 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:40.029089    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wwhv" event={"ID":"4fbfe24c-0bfe-408e-9be5-03ca2a14acec","Type":"ContainerDied","Data":"537055ac6beebe310a12a747df1c7c49580ec0b4fd5008c0d9b969932e70adc3"}
Jul 30 19:32:40.029565 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:40.029108    2554 scope.go:117] "RemoveContainer" containerID="64d2644f084b14ef6ea44aff178b45b2a0d558ee02ce5a4b95cc8bdda34f5ba6"
Jul 30 19:32:40.032247 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:40.032214    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lzkrl" event={"ID":"d4b8740b-ed13-4da7-84cc-b60b33f80e54","Type":"ContainerStarted","Data":"b9cd3a7bd7f73d290afff4ea03b683823409a884fc1eff7c01fc5ea808ed2ef1"}
Jul 30 19:32:40.032374 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:40.032270    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lzkrl" event={"ID":"d4b8740b-ed13-4da7-84cc-b60b33f80e54","Type":"ContainerStarted","Data":"3512850fc9492164481bb504b0e779b4fafaeae49cf8f8ecd78db57c81b84e1c"}
Jul 30 19:32:40.032374 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:40.032329    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lzkrl"
Jul 30 19:32:40.038783 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:40.038762    2554 scope.go:117] "RemoveContainer" containerID="64d2644f084b14ef6ea44aff178b45b2a0d558ee02ce5a4b95cc8bdda34f5ba6"
Jul 30 19:32:40.039092 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:32:40.039073    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64d2644f084b14ef6ea44aff178b45b2a0d558ee02ce5a4b95cc8bdda34f5ba6\": container with ID starting with 64d2644f084b14ef6ea44aff178b45b2a0d558ee02ce5a4b95cc8bdda34f5ba6 not found: ID does not exist" containerID="64d2644f084b14ef6ea44aff178b45b2a0d558ee02ce5a4b95cc8bdda34f5ba6"
Jul 30 19:32:40.039152 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:40.039102    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64d2644f084b14ef6ea44aff178b45b2a0d558ee02ce5a4b95cc8bdda34f5ba6"} err="failed to get container status \"64d2644f084b14ef6ea44aff178b45b2a0d558ee02ce5a4b95cc8bdda34f5ba6\": rpc error: code = NotFound desc = could not find container \"64d2644f084b14ef6ea44aff178b45b2a0d558ee02ce5a4b95cc8bdda34f5ba6\": container with ID starting with 64d2644f084b14ef6ea44aff178b45b2a0d558ee02ce5a4b95cc8bdda34f5ba6 not found: ID does not exist"
Jul 30 19:32:40.055473 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:40.055425    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lzkrl" podStartSLOduration=1.055408438 podStartE2EDuration="1.055408438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:32:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:32:40.053521287 +0000 UTC m=+488.018102371" watchObservedRunningTime="2026-07-30 19:32:40.055408438 +0000 UTC m=+488.019989519"
Jul 30 19:32:40.067763 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:40.067730    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wwhv"]
Jul 30 19:32:40.073129 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:40.073103    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4wwhv"]
Jul 30 19:32:40.487089 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:40.487054    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fbfe24c-0bfe-408e-9be5-03ca2a14acec" path="/var/lib/kubelet/pods/4fbfe24c-0bfe-408e-9be5-03ca2a14acec/volumes"
Jul 30 19:32:40.715982 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:40.715940    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:40.819732 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:40.819641    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:40.916356 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:40.916326    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:41.361014 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:41.360977    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:41.461594 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:41.461554    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:42.632708 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:42.632670    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:42.733933 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:42.733902    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:43.447116 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:43.447080    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:43.549992 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:43.549937    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:44.755790 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:44.755749    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:46.840930 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:46.840897    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:46.941583 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:46.941545    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:50.242925 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:50.242876    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:50.309526 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:50.309479    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:51.041607 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:51.041575    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lzkrl"
Jul 30 19:32:51.092018 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:51.091980    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86prw"]
Jul 30 19:32:51.092280 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:51.092233    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86prw" podUID="b6a6f617-7c9d-497d-b347-7ea4c0dd3b24" containerName="manager" containerID="cri-o://644e17ba5bdc7089fd45560d19c56a0525878c70ad2bdb92f3bce8c7de976f7b" gracePeriod=10
Jul 30 19:32:51.371215 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:51.371176    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86prw"
Jul 30 19:32:51.484344 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:51.484308    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwbqq\" (UniqueName: \"kubernetes.io/projected/b6a6f617-7c9d-497d-b347-7ea4c0dd3b24-kube-api-access-bwbqq\") pod \"b6a6f617-7c9d-497d-b347-7ea4c0dd3b24\" (UID: \"b6a6f617-7c9d-497d-b347-7ea4c0dd3b24\") "
Jul 30 19:32:51.484515 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:51.484358    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b6a6f617-7c9d-497d-b347-7ea4c0dd3b24-extensions-socket-volume\") pod \"b6a6f617-7c9d-497d-b347-7ea4c0dd3b24\" (UID: \"b6a6f617-7c9d-497d-b347-7ea4c0dd3b24\") "
Jul 30 19:32:51.484754 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:51.484731    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6a6f617-7c9d-497d-b347-7ea4c0dd3b24-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b6a6f617-7c9d-497d-b347-7ea4c0dd3b24" (UID: "b6a6f617-7c9d-497d-b347-7ea4c0dd3b24"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:32:51.486633 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:51.486609    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6a6f617-7c9d-497d-b347-7ea4c0dd3b24-kube-api-access-bwbqq" (OuterVolumeSpecName: "kube-api-access-bwbqq") pod "b6a6f617-7c9d-497d-b347-7ea4c0dd3b24" (UID: "b6a6f617-7c9d-497d-b347-7ea4c0dd3b24"). InnerVolumeSpecName "kube-api-access-bwbqq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:32:51.586009 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:51.585934    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwbqq\" (UniqueName: \"kubernetes.io/projected/b6a6f617-7c9d-497d-b347-7ea4c0dd3b24-kube-api-access-bwbqq\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:32:51.586009 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:51.586000    2554 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b6a6f617-7c9d-497d-b347-7ea4c0dd3b24-extensions-socket-volume\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:32:52.100994 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:52.100929    2554 generic.go:356] "Generic (PLEG): container finished" podID="b6a6f617-7c9d-497d-b347-7ea4c0dd3b24" containerID="644e17ba5bdc7089fd45560d19c56a0525878c70ad2bdb92f3bce8c7de976f7b" exitCode=0
Jul 30 19:32:52.100994 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:52.100997    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86prw"
Jul 30 19:32:52.101204 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:52.101016    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86prw" event={"ID":"b6a6f617-7c9d-497d-b347-7ea4c0dd3b24","Type":"ContainerDied","Data":"644e17ba5bdc7089fd45560d19c56a0525878c70ad2bdb92f3bce8c7de976f7b"}
Jul 30 19:32:52.101204 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:52.101073    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86prw" event={"ID":"b6a6f617-7c9d-497d-b347-7ea4c0dd3b24","Type":"ContainerDied","Data":"b08a0ff1b241715c6b0ed0b76f2605b75e8fadb9366f9e74cc169a86f36319be"}
Jul 30 19:32:52.101204 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:52.101095    2554 scope.go:117] "RemoveContainer" containerID="644e17ba5bdc7089fd45560d19c56a0525878c70ad2bdb92f3bce8c7de976f7b"
Jul 30 19:32:52.110811 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:52.110794    2554 scope.go:117] "RemoveContainer" containerID="644e17ba5bdc7089fd45560d19c56a0525878c70ad2bdb92f3bce8c7de976f7b"
Jul 30 19:32:52.111170 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:32:52.111145    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"644e17ba5bdc7089fd45560d19c56a0525878c70ad2bdb92f3bce8c7de976f7b\": container with ID starting with 644e17ba5bdc7089fd45560d19c56a0525878c70ad2bdb92f3bce8c7de976f7b not found: ID does not exist" containerID="644e17ba5bdc7089fd45560d19c56a0525878c70ad2bdb92f3bce8c7de976f7b"
Jul 30 19:32:52.111312 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:52.111181    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"644e17ba5bdc7089fd45560d19c56a0525878c70ad2bdb92f3bce8c7de976f7b"} err="failed to get container status \"644e17ba5bdc7089fd45560d19c56a0525878c70ad2bdb92f3bce8c7de976f7b\": rpc error: code = NotFound desc = could not find container \"644e17ba5bdc7089fd45560d19c56a0525878c70ad2bdb92f3bce8c7de976f7b\": container with ID starting with 644e17ba5bdc7089fd45560d19c56a0525878c70ad2bdb92f3bce8c7de976f7b not found: ID does not exist"
Jul 30 19:32:52.123634 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:52.123597    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86prw"]
Jul 30 19:32:52.127205 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:52.127178    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-86prw"]
Jul 30 19:32:52.486593 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:52.486560    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6a6f617-7c9d-497d-b347-7ea4c0dd3b24" path="/var/lib/kubelet/pods/b6a6f617-7c9d-497d-b347-7ea4c0dd3b24/volumes"
Jul 30 19:32:55.362907 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.362870    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"]
Jul 30 19:32:55.363325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.363245    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fbfe24c-0bfe-408e-9be5-03ca2a14acec" containerName="manager"
Jul 30 19:32:55.363325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.363258    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fbfe24c-0bfe-408e-9be5-03ca2a14acec" containerName="manager"
Jul 30 19:32:55.363325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.363283    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6a6f617-7c9d-497d-b347-7ea4c0dd3b24" containerName="manager"
Jul 30 19:32:55.363325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.363288    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6a6f617-7c9d-497d-b347-7ea4c0dd3b24" containerName="manager"
Jul 30 19:32:55.363479 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.363343    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6a6f617-7c9d-497d-b347-7ea4c0dd3b24" containerName="manager"
Jul 30 19:32:55.363479 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.363353    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fbfe24c-0bfe-408e-9be5-03ca2a14acec" containerName="manager"
Jul 30 19:32:55.388189 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.388135    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:55.423841 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.423797    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"]
Jul 30 19:32:55.424041 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.423921    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.426723 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.426697    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-b2xkc\""
Jul 30 19:32:55.517981 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.517933    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0582054c-2090-4cc6-9420-50011ca2d94f-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fbm6v\" (UID: \"0582054c-2090-4cc6-9420-50011ca2d94f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.518161 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.517989    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0582054c-2090-4cc6-9420-50011ca2d94f-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fbm6v\" (UID: \"0582054c-2090-4cc6-9420-50011ca2d94f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.518161 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.518046    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6td5q\" (UniqueName: \"kubernetes.io/projected/0582054c-2090-4cc6-9420-50011ca2d94f-kube-api-access-6td5q\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fbm6v\" (UID: \"0582054c-2090-4cc6-9420-50011ca2d94f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.518161 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.518108    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0582054c-2090-4cc6-9420-50011ca2d94f-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fbm6v\" (UID: \"0582054c-2090-4cc6-9420-50011ca2d94f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.518161 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.518134    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0582054c-2090-4cc6-9420-50011ca2d94f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fbm6v\" (UID: \"0582054c-2090-4cc6-9420-50011ca2d94f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.518299 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.518171    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0582054c-2090-4cc6-9420-50011ca2d94f-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fbm6v\" (UID: \"0582054c-2090-4cc6-9420-50011ca2d94f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.518299 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.518219    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0582054c-2090-4cc6-9420-50011ca2d94f-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fbm6v\" (UID: \"0582054c-2090-4cc6-9420-50011ca2d94f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.518299 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.518250    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0582054c-2090-4cc6-9420-50011ca2d94f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fbm6v\" (UID: \"0582054c-2090-4cc6-9420-50011ca2d94f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.518299 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.518290    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0582054c-2090-4cc6-9420-50011ca2d94f-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fbm6v\" (UID: \"0582054c-2090-4cc6-9420-50011ca2d94f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.619574 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.619493    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0582054c-2090-4cc6-9420-50011ca2d94f-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fbm6v\" (UID: \"0582054c-2090-4cc6-9420-50011ca2d94f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.619574 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.619526    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0582054c-2090-4cc6-9420-50011ca2d94f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fbm6v\" (UID: \"0582054c-2090-4cc6-9420-50011ca2d94f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.619821 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.619580    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0582054c-2090-4cc6-9420-50011ca2d94f-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fbm6v\" (UID: \"0582054c-2090-4cc6-9420-50011ca2d94f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.619821 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.619794    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0582054c-2090-4cc6-9420-50011ca2d94f-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fbm6v\" (UID: \"0582054c-2090-4cc6-9420-50011ca2d94f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.620023 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.619947    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0582054c-2090-4cc6-9420-50011ca2d94f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fbm6v\" (UID: \"0582054c-2090-4cc6-9420-50011ca2d94f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.620113 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.620032    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0582054c-2090-4cc6-9420-50011ca2d94f-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fbm6v\" (UID: \"0582054c-2090-4cc6-9420-50011ca2d94f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.620168 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.620141    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0582054c-2090-4cc6-9420-50011ca2d94f-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fbm6v\" (UID: \"0582054c-2090-4cc6-9420-50011ca2d94f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.620211 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.620143    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0582054c-2090-4cc6-9420-50011ca2d94f-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fbm6v\" (UID: \"0582054c-2090-4cc6-9420-50011ca2d94f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.620256 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.620226    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0582054c-2090-4cc6-9420-50011ca2d94f-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fbm6v\" (UID: \"0582054c-2090-4cc6-9420-50011ca2d94f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.620293 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.620266    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0582054c-2090-4cc6-9420-50011ca2d94f-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fbm6v\" (UID: \"0582054c-2090-4cc6-9420-50011ca2d94f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.620350 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.620297    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6td5q\" (UniqueName: \"kubernetes.io/projected/0582054c-2090-4cc6-9420-50011ca2d94f-kube-api-access-6td5q\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fbm6v\" (UID: \"0582054c-2090-4cc6-9420-50011ca2d94f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.620486 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.620467    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0582054c-2090-4cc6-9420-50011ca2d94f-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fbm6v\" (UID: \"0582054c-2090-4cc6-9420-50011ca2d94f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.620565 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.620542    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0582054c-2090-4cc6-9420-50011ca2d94f-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fbm6v\" (UID: \"0582054c-2090-4cc6-9420-50011ca2d94f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.620666 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.620646    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0582054c-2090-4cc6-9420-50011ca2d94f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fbm6v\" (UID: \"0582054c-2090-4cc6-9420-50011ca2d94f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.622240 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.622219    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0582054c-2090-4cc6-9420-50011ca2d94f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fbm6v\" (UID: \"0582054c-2090-4cc6-9420-50011ca2d94f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.622353 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.622335    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0582054c-2090-4cc6-9420-50011ca2d94f-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fbm6v\" (UID: \"0582054c-2090-4cc6-9420-50011ca2d94f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.627988 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.627943    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0582054c-2090-4cc6-9420-50011ca2d94f-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fbm6v\" (UID: \"0582054c-2090-4cc6-9420-50011ca2d94f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.628129 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.628110    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6td5q\" (UniqueName: \"kubernetes.io/projected/0582054c-2090-4cc6-9420-50011ca2d94f-kube-api-access-6td5q\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fbm6v\" (UID: \"0582054c-2090-4cc6-9420-50011ca2d94f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.731539 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.731485    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:55.734372 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.734351    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:55.881967 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.881919    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"]
Jul 30 19:32:55.883161 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:32:55.883135    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0582054c_2090_4cc6_9420_50011ca2d94f.slice/crio-20f31aa43cc63c5fb6a402332fae6c8bb19b6392907565ef853736146db04c25 WatchSource:0}: Error finding container 20f31aa43cc63c5fb6a402332fae6c8bb19b6392907565ef853736146db04c25: Status 404 returned error can't find the container with id 20f31aa43cc63c5fb6a402332fae6c8bb19b6392907565ef853736146db04c25
Jul 30 19:32:55.885342 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.885308    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 19:32:55.885451 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.885393    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 19:32:55.885451 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:55.885440    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 19:32:56.125629 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:56.125589    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v" event={"ID":"0582054c-2090-4cc6-9420-50011ca2d94f","Type":"ContainerStarted","Data":"56ebb206d02a6625bc0b7b7f7427790e59d4a6e40fb16dc699da18063a3c8679"}
Jul 30 19:32:56.126077 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:56.125661    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v" event={"ID":"0582054c-2090-4cc6-9420-50011ca2d94f","Type":"ContainerStarted","Data":"20f31aa43cc63c5fb6a402332fae6c8bb19b6392907565ef853736146db04c25"}
Jul 30 19:32:56.146121 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:56.146018    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v" podStartSLOduration=1.146002027 podStartE2EDuration="1.146002027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:32:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:32:56.143907037 +0000 UTC m=+504.108488127" watchObservedRunningTime="2026-07-30 19:32:56.146002027 +0000 UTC m=+504.110583123"
Jul 30 19:32:56.734750 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:56.734714    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:56.739780 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:56.739746    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:57.132162 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:57.132074    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:57.133246 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:57.133224    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:32:58.809076 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:58.809042    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:32:58.907635 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:32:58.907560    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:33:00.123470 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:00.123400    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:33:12.159602 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:12.159560    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:33:12.173414 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:12.173386    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qwjwm"]
Jul 30 19:33:12.256683 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:12.256653    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qwjwm"]
Jul 30 19:33:12.256869 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:12.256774    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qwjwm"
Jul 30 19:33:12.259366 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:12.259345    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 30 19:33:12.264441 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:12.264412    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qwjwm"]
Jul 30 19:33:12.276171 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:12.276141    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:33:12.347791 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:12.347749    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:33:12.369103 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:12.369055    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwq88\" (UniqueName: \"kubernetes.io/projected/c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe-kube-api-access-xwq88\") pod \"limitador-limitador-64c94fd64f-qwjwm\" (UID: \"c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qwjwm"
Jul 30 19:33:12.369309 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:12.369194    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe-config-file\") pod \"limitador-limitador-64c94fd64f-qwjwm\" (UID: \"c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qwjwm"
Jul 30 19:33:12.470346 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:12.470307    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwq88\" (UniqueName: \"kubernetes.io/projected/c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe-kube-api-access-xwq88\") pod \"limitador-limitador-64c94fd64f-qwjwm\" (UID: \"c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qwjwm"
Jul 30 19:33:12.470521 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:12.470369    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe-config-file\") pod \"limitador-limitador-64c94fd64f-qwjwm\" (UID: \"c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qwjwm"
Jul 30 19:33:12.470996 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:12.470978    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe-config-file\") pod \"limitador-limitador-64c94fd64f-qwjwm\" (UID: \"c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qwjwm"
Jul 30 19:33:12.482384 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:12.482351    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwq88\" (UniqueName: \"kubernetes.io/projected/c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe-kube-api-access-xwq88\") pod \"limitador-limitador-64c94fd64f-qwjwm\" (UID: \"c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qwjwm"
Jul 30 19:33:12.569251 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:12.569222    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qwjwm"
Jul 30 19:33:12.708793 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:12.708768    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qwjwm"]
Jul 30 19:33:12.710636 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:33:12.710605    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4fbfc9b_f7c9_4d9c_8fda_c5b9fc44a8fe.slice/crio-a7f9fd2f0506b90e0187097af08c39b87a25a0afdc76fa7ee6ec0c0b99cdd8ec WatchSource:0}: Error finding container a7f9fd2f0506b90e0187097af08c39b87a25a0afdc76fa7ee6ec0c0b99cdd8ec: Status 404 returned error can't find the container with id a7f9fd2f0506b90e0187097af08c39b87a25a0afdc76fa7ee6ec0c0b99cdd8ec
Jul 30 19:33:12.985713 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:12.985673    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:33:13.045007 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:13.044972    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:33:13.219665 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:13.219626    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qwjwm" event={"ID":"c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe","Type":"ContainerStarted","Data":"a7f9fd2f0506b90e0187097af08c39b87a25a0afdc76fa7ee6ec0c0b99cdd8ec"}
Jul 30 19:33:16.239496 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:16.239460    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qwjwm" event={"ID":"c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe","Type":"ContainerStarted","Data":"04af602f625e3976e87e0dc78218a7b976e56cc2431d9c7921d0f38d0e7c56b3"}
Jul 30 19:33:16.239931 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:16.239549    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-qwjwm"
Jul 30 19:33:16.258071 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:16.258014    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-qwjwm" podStartSLOduration=1.463408464 podStartE2EDuration="4.257996886s" totalImagesPullingTime="2.794588422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:33:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:33:16.256479844 +0000 UTC m=+524.221060926" watchObservedRunningTime="2026-07-30 19:33:16.257996886 +0000 UTC m=+524.222577967"
Jul 30 19:33:27.245776 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:27.245687    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-qwjwm"
Jul 30 19:33:29.601862 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:29.601825    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qwjwm"]
Jul 30 19:33:29.602368 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:29.602061    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-qwjwm" podUID="c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe" containerName="limitador" containerID="cri-o://04af602f625e3976e87e0dc78218a7b976e56cc2431d9c7921d0f38d0e7c56b3" gracePeriod=30
Jul 30 19:33:30.155823 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:30.155796    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qwjwm"
Jul 30 19:33:30.232934 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:30.232899    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xwq88\" (UniqueName: \"kubernetes.io/projected/c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe-kube-api-access-xwq88\") pod \"c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe\" (UID: \"c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe\") "
Jul 30 19:33:30.233157 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:30.233050    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe-config-file\") pod \"c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe\" (UID: \"c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe\") "
Jul 30 19:33:30.233399 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:30.233374    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe-config-file" (OuterVolumeSpecName: "config-file") pod "c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe" (UID: "c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 19:33:30.235137 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:30.235112    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe-kube-api-access-xwq88" (OuterVolumeSpecName: "kube-api-access-xwq88") pod "c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe" (UID: "c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe"). InnerVolumeSpecName "kube-api-access-xwq88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:33:30.316634 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:30.316595    2554 generic.go:356] "Generic (PLEG): container finished" podID="c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe" containerID="04af602f625e3976e87e0dc78218a7b976e56cc2431d9c7921d0f38d0e7c56b3" exitCode=0
Jul 30 19:33:30.316819 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:30.316640    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qwjwm"
Jul 30 19:33:30.316819 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:30.316681    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qwjwm" event={"ID":"c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe","Type":"ContainerDied","Data":"04af602f625e3976e87e0dc78218a7b976e56cc2431d9c7921d0f38d0e7c56b3"}
Jul 30 19:33:30.316819 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:30.316734    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qwjwm" event={"ID":"c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe","Type":"ContainerDied","Data":"a7f9fd2f0506b90e0187097af08c39b87a25a0afdc76fa7ee6ec0c0b99cdd8ec"}
Jul 30 19:33:30.316819 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:30.316759    2554 scope.go:117] "RemoveContainer" containerID="04af602f625e3976e87e0dc78218a7b976e56cc2431d9c7921d0f38d0e7c56b3"
Jul 30 19:33:30.326944 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:30.326926    2554 scope.go:117] "RemoveContainer" containerID="04af602f625e3976e87e0dc78218a7b976e56cc2431d9c7921d0f38d0e7c56b3"
Jul 30 19:33:30.327324 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:33:30.327303    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04af602f625e3976e87e0dc78218a7b976e56cc2431d9c7921d0f38d0e7c56b3\": container with ID starting with 04af602f625e3976e87e0dc78218a7b976e56cc2431d9c7921d0f38d0e7c56b3 not found: ID does not exist" containerID="04af602f625e3976e87e0dc78218a7b976e56cc2431d9c7921d0f38d0e7c56b3"
Jul 30 19:33:30.327408 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:30.327330    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04af602f625e3976e87e0dc78218a7b976e56cc2431d9c7921d0f38d0e7c56b3"} err="failed to get container status \"04af602f625e3976e87e0dc78218a7b976e56cc2431d9c7921d0f38d0e7c56b3\": rpc error: code = NotFound desc = could not find container \"04af602f625e3976e87e0dc78218a7b976e56cc2431d9c7921d0f38d0e7c56b3\": container with ID starting with 04af602f625e3976e87e0dc78218a7b976e56cc2431d9c7921d0f38d0e7c56b3 not found: ID does not exist"
Jul 30 19:33:30.334134 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:30.334113    2554 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe-config-file\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:33:30.334134 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:30.334134    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwq88\" (UniqueName: \"kubernetes.io/projected/c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe-kube-api-access-xwq88\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:33:30.339898 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:30.339870    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qwjwm"]
Jul 30 19:33:30.344589 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:30.344566    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qwjwm"]
Jul 30 19:33:30.486689 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:30.486598    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe" path="/var/lib/kubelet/pods/c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe/volumes"
Jul 30 19:33:33.745147 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:33.745104    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:33:33.844898 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:33.844853    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:33:34.223119 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:34.223084    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8977m"]
Jul 30 19:33:34.223535 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:34.223512    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe" containerName="limitador"
Jul 30 19:33:34.223535 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:34.223533    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe" containerName="limitador"
Jul 30 19:33:34.223756 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:34.223698    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4fbfc9b-f7c9-4d9c-8fda-c5b9fc44a8fe" containerName="limitador"
Jul 30 19:33:34.228202 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:34.228177    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8977m"
Jul 30 19:33:34.230567 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:34.230542    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-pknx2\""
Jul 30 19:33:34.230724 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:34.230702    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 19:33:34.231736 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:34.231714    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 30 19:33:34.231862 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:34.231831    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 30 19:33:34.235348 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:34.235320    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8977m"]
Jul 30 19:33:34.258282 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:34.258248    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:33:34.373066 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:34.373029    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf9ff\" (UniqueName: \"kubernetes.io/projected/ca6f2c34-27da-488d-a73e-27f0cf705c92-kube-api-access-gf9ff\") pod \"postgres-69fb4594cf-8977m\" (UID: \"ca6f2c34-27da-488d-a73e-27f0cf705c92\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8977m"
Jul 30 19:33:34.373256 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:34.373083    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ca6f2c34-27da-488d-a73e-27f0cf705c92-data\") pod \"postgres-69fb4594cf-8977m\" (UID: \"ca6f2c34-27da-488d-a73e-27f0cf705c92\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8977m"
Jul 30 19:33:34.473904 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:34.473805    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf9ff\" (UniqueName: \"kubernetes.io/projected/ca6f2c34-27da-488d-a73e-27f0cf705c92-kube-api-access-gf9ff\") pod \"postgres-69fb4594cf-8977m\" (UID: \"ca6f2c34-27da-488d-a73e-27f0cf705c92\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8977m"
Jul 30 19:33:34.473904 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:34.473852    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ca6f2c34-27da-488d-a73e-27f0cf705c92-data\") pod \"postgres-69fb4594cf-8977m\" (UID: \"ca6f2c34-27da-488d-a73e-27f0cf705c92\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8977m"
Jul 30 19:33:34.474260 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:34.474241    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ca6f2c34-27da-488d-a73e-27f0cf705c92-data\") pod \"postgres-69fb4594cf-8977m\" (UID: \"ca6f2c34-27da-488d-a73e-27f0cf705c92\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8977m"
Jul 30 19:33:34.481834 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:34.481808    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf9ff\" (UniqueName: \"kubernetes.io/projected/ca6f2c34-27da-488d-a73e-27f0cf705c92-kube-api-access-gf9ff\") pod \"postgres-69fb4594cf-8977m\" (UID: \"ca6f2c34-27da-488d-a73e-27f0cf705c92\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8977m"
Jul 30 19:33:34.542317 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:34.542283    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8977m"
Jul 30 19:33:34.678240 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:34.678203    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8977m"]
Jul 30 19:33:34.680095 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:33:34.680069    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca6f2c34_27da_488d_a73e_27f0cf705c92.slice/crio-5dbae3f8e479dd6227cd08dca9760687927fff4e368c7073a7861e8a6fa62bc2 WatchSource:0}: Error finding container 5dbae3f8e479dd6227cd08dca9760687927fff4e368c7073a7861e8a6fa62bc2: Status 404 returned error can't find the container with id 5dbae3f8e479dd6227cd08dca9760687927fff4e368c7073a7861e8a6fa62bc2
Jul 30 19:33:35.345866 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:35.345826    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8977m" event={"ID":"ca6f2c34-27da-488d-a73e-27f0cf705c92","Type":"ContainerStarted","Data":"5dbae3f8e479dd6227cd08dca9760687927fff4e368c7073a7861e8a6fa62bc2"}
Jul 30 19:33:40.380195 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:40.380082    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8977m" event={"ID":"ca6f2c34-27da-488d-a73e-27f0cf705c92","Type":"ContainerStarted","Data":"706565de3bf73b9021ba9cfb2ec2c619fa2372c9d80e5b9739b4e965c39e740c"}
Jul 30 19:33:40.380195 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:40.380157    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8977m"
Jul 30 19:33:40.397530 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:40.397480    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8977m" podStartSLOduration=1.026112396 podStartE2EDuration="6.397459844s" totalImagesPullingTime="5.371347448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:33:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:33:40.395537945 +0000 UTC m=+548.360119039" watchObservedRunningTime="2026-07-30 19:33:40.397459844 +0000 UTC m=+548.362040983"
Jul 30 19:33:46.414855 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:46.414824    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8977m"
Jul 30 19:33:46.752607 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:46.752576    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:33:47.237218 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:47.237178    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:33:53.375152 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:53.375104    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:33:53.476644 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:53.476604    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:33:53.560636 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:53.560603    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:33:53.564925 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:53.564891    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-785db54bd5-hjlpt"]
Jul 30 19:33:53.573339 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:53.573313    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-785db54bd5-hjlpt"
Jul 30 19:33:53.576622 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:53.576584    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6n8w5\""
Jul 30 19:33:53.576789 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:53.576626    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 30 19:33:53.576789 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:53.576585    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 30 19:33:53.577864 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:53.577840    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-785db54bd5-hjlpt"]
Jul 30 19:33:53.642005 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:53.641883    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmw82\" (UniqueName: \"kubernetes.io/projected/c7c791b0-e5e2-4cd2-991a-312f43fce85a-kube-api-access-lmw82\") pod \"maas-controller-785db54bd5-hjlpt\" (UID: \"c7c791b0-e5e2-4cd2-991a-312f43fce85a\") " pod="opendatahub/maas-controller-785db54bd5-hjlpt"
Jul 30 19:33:53.642185 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:53.642038    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c7c791b0-e5e2-4cd2-991a-312f43fce85a-webhook-cert\") pod \"maas-controller-785db54bd5-hjlpt\" (UID: \"c7c791b0-e5e2-4cd2-991a-312f43fce85a\") " pod="opendatahub/maas-controller-785db54bd5-hjlpt"
Jul 30 19:33:53.742508 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:53.742467    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmw82\" (UniqueName: \"kubernetes.io/projected/c7c791b0-e5e2-4cd2-991a-312f43fce85a-kube-api-access-lmw82\") pod \"maas-controller-785db54bd5-hjlpt\" (UID: \"c7c791b0-e5e2-4cd2-991a-312f43fce85a\") " pod="opendatahub/maas-controller-785db54bd5-hjlpt"
Jul 30 19:33:53.742686 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:53.742565    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c7c791b0-e5e2-4cd2-991a-312f43fce85a-webhook-cert\") pod \"maas-controller-785db54bd5-hjlpt\" (UID: \"c7c791b0-e5e2-4cd2-991a-312f43fce85a\") " pod="opendatahub/maas-controller-785db54bd5-hjlpt"
Jul 30 19:33:53.745125 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:53.745100    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c7c791b0-e5e2-4cd2-991a-312f43fce85a-webhook-cert\") pod \"maas-controller-785db54bd5-hjlpt\" (UID: \"c7c791b0-e5e2-4cd2-991a-312f43fce85a\") " pod="opendatahub/maas-controller-785db54bd5-hjlpt"
Jul 30 19:33:53.750522 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:53.750493    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmw82\" (UniqueName: \"kubernetes.io/projected/c7c791b0-e5e2-4cd2-991a-312f43fce85a-kube-api-access-lmw82\") pod \"maas-controller-785db54bd5-hjlpt\" (UID: \"c7c791b0-e5e2-4cd2-991a-312f43fce85a\") " pod="opendatahub/maas-controller-785db54bd5-hjlpt"
Jul 30 19:33:53.885113 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:53.885072    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-785db54bd5-hjlpt"
Jul 30 19:33:54.021606 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:54.021576    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-785db54bd5-hjlpt"]
Jul 30 19:33:54.023566 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:33:54.023536    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7c791b0_e5e2_4cd2_991a_312f43fce85a.slice/crio-1681ecfcd2c2e98ffa7d3e8a78aaf3058d22a7f5e2e96cdacfe672019788fbfc WatchSource:0}: Error finding container 1681ecfcd2c2e98ffa7d3e8a78aaf3058d22a7f5e2e96cdacfe672019788fbfc: Status 404 returned error can't find the container with id 1681ecfcd2c2e98ffa7d3e8a78aaf3058d22a7f5e2e96cdacfe672019788fbfc
Jul 30 19:33:54.463171 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:54.463123    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-785db54bd5-hjlpt" event={"ID":"c7c791b0-e5e2-4cd2-991a-312f43fce85a","Type":"ContainerStarted","Data":"1681ecfcd2c2e98ffa7d3e8a78aaf3058d22a7f5e2e96cdacfe672019788fbfc"}
Jul 30 19:33:56.231781 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:56.231746    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:33:56.335880 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:56.335841    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:33:56.477405 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:56.477365    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-785db54bd5-hjlpt" event={"ID":"c7c791b0-e5e2-4cd2-991a-312f43fce85a","Type":"ContainerStarted","Data":"2f1d1ef596760d7b487def8e816cdc4f5bcefb2548800fec1116b314bcba0721"}
Jul 30 19:33:56.477606 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:56.477456    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-785db54bd5-hjlpt"
Jul 30 19:33:56.520132 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:33:56.520080    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-785db54bd5-hjlpt" podStartSLOduration=1.347691952 podStartE2EDuration="3.520062613s" totalImagesPullingTime="2.172370661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:33:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:33:56.517898093 +0000 UTC m=+564.482479175" watchObservedRunningTime="2026-07-30 19:33:56.520062613 +0000 UTC m=+564.484643695"
Jul 30 19:34:07.485200 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:07.485166    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-785db54bd5-hjlpt"
Jul 30 19:34:27.994816 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:27.994775    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:34:28.093022 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:28.092988    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:34:29.506221 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:29.506184    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:34:29.605066 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:29.605031    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:34:29.815591 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:29.815504    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-844cc9dc96-gs8n7"]
Jul 30 19:34:29.820547 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:29.820522    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-844cc9dc96-gs8n7"
Jul 30 19:34:29.823866 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:29.823845    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 30 19:34:29.824013 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:29.823847    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6qhn5\""
Jul 30 19:34:29.832201 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:29.832179    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-844cc9dc96-gs8n7"]
Jul 30 19:34:29.976158 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:29.976116    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dafdfcb4-7499-42ec-b9ed-5856bc2f5eaa-openshift-service-ca\") pod \"maas-api-844cc9dc96-gs8n7\" (UID: \"dafdfcb4-7499-42ec-b9ed-5856bc2f5eaa\") " pod="odh-ai-gateway-infra/maas-api-844cc9dc96-gs8n7"
Jul 30 19:34:29.976158 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:29.976163    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dafdfcb4-7499-42ec-b9ed-5856bc2f5eaa-maas-api-tls\") pod \"maas-api-844cc9dc96-gs8n7\" (UID: \"dafdfcb4-7499-42ec-b9ed-5856bc2f5eaa\") " pod="odh-ai-gateway-infra/maas-api-844cc9dc96-gs8n7"
Jul 30 19:34:29.976383 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:29.976217    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhn5k\" (UniqueName: \"kubernetes.io/projected/dafdfcb4-7499-42ec-b9ed-5856bc2f5eaa-kube-api-access-vhn5k\") pod \"maas-api-844cc9dc96-gs8n7\" (UID: \"dafdfcb4-7499-42ec-b9ed-5856bc2f5eaa\") " pod="odh-ai-gateway-infra/maas-api-844cc9dc96-gs8n7"
Jul 30 19:34:30.077440 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:30.077344    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dafdfcb4-7499-42ec-b9ed-5856bc2f5eaa-openshift-service-ca\") pod \"maas-api-844cc9dc96-gs8n7\" (UID: \"dafdfcb4-7499-42ec-b9ed-5856bc2f5eaa\") " pod="odh-ai-gateway-infra/maas-api-844cc9dc96-gs8n7"
Jul 30 19:34:30.077440 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:30.077397    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dafdfcb4-7499-42ec-b9ed-5856bc2f5eaa-maas-api-tls\") pod \"maas-api-844cc9dc96-gs8n7\" (UID: \"dafdfcb4-7499-42ec-b9ed-5856bc2f5eaa\") " pod="odh-ai-gateway-infra/maas-api-844cc9dc96-gs8n7"
Jul 30 19:34:30.077646 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:30.077451    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhn5k\" (UniqueName: \"kubernetes.io/projected/dafdfcb4-7499-42ec-b9ed-5856bc2f5eaa-kube-api-access-vhn5k\") pod \"maas-api-844cc9dc96-gs8n7\" (UID: \"dafdfcb4-7499-42ec-b9ed-5856bc2f5eaa\") " pod="odh-ai-gateway-infra/maas-api-844cc9dc96-gs8n7"
Jul 30 19:34:30.078037 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:30.078017    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dafdfcb4-7499-42ec-b9ed-5856bc2f5eaa-openshift-service-ca\") pod \"maas-api-844cc9dc96-gs8n7\" (UID: \"dafdfcb4-7499-42ec-b9ed-5856bc2f5eaa\") " pod="odh-ai-gateway-infra/maas-api-844cc9dc96-gs8n7"
Jul 30 19:34:30.079906 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:30.079889    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dafdfcb4-7499-42ec-b9ed-5856bc2f5eaa-maas-api-tls\") pod \"maas-api-844cc9dc96-gs8n7\" (UID: \"dafdfcb4-7499-42ec-b9ed-5856bc2f5eaa\") " pod="odh-ai-gateway-infra/maas-api-844cc9dc96-gs8n7"
Jul 30 19:34:30.087501 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:30.087471    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhn5k\" (UniqueName: \"kubernetes.io/projected/dafdfcb4-7499-42ec-b9ed-5856bc2f5eaa-kube-api-access-vhn5k\") pod \"maas-api-844cc9dc96-gs8n7\" (UID: \"dafdfcb4-7499-42ec-b9ed-5856bc2f5eaa\") " pod="odh-ai-gateway-infra/maas-api-844cc9dc96-gs8n7"
Jul 30 19:34:30.132126 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:30.132089    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-844cc9dc96-gs8n7"
Jul 30 19:34:30.192874 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:30.192845    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:34:30.283659 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:30.283620    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-844cc9dc96-gs8n7"]
Jul 30 19:34:30.284655 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:30.284628    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:34:30.285501 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:34:30.285478    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddafdfcb4_7499_42ec_b9ed_5856bc2f5eaa.slice/crio-32149a19f67caf89d9b44a874a036ebda1881afde977a568c457a46b02c836b3 WatchSource:0}: Error finding container 32149a19f67caf89d9b44a874a036ebda1881afde977a568c457a46b02c836b3: Status 404 returned error can't find the container with id 32149a19f67caf89d9b44a874a036ebda1881afde977a568c457a46b02c836b3
Jul 30 19:34:30.660028 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:30.659988    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-844cc9dc96-gs8n7" event={"ID":"dafdfcb4-7499-42ec-b9ed-5856bc2f5eaa","Type":"ContainerStarted","Data":"32149a19f67caf89d9b44a874a036ebda1881afde977a568c457a46b02c836b3"}
Jul 30 19:34:32.445243 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:32.445203    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/1.log"
Jul 30 19:34:32.445666 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:32.445555    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/1.log"
Jul 30 19:34:32.453902 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:32.453876    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 19:34:32.454189 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:32.454167    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 19:34:32.674562 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:32.674469    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-844cc9dc96-gs8n7" event={"ID":"dafdfcb4-7499-42ec-b9ed-5856bc2f5eaa","Type":"ContainerStarted","Data":"9c04c000e4c63a0d0b02e1be15c8fa6a46412d6204bc05cfc2d32e092b02b649"}
Jul 30 19:34:32.674699 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:32.674573    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-844cc9dc96-gs8n7"
Jul 30 19:34:32.692934 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:32.692880    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-844cc9dc96-gs8n7" podStartSLOduration=1.642654572 podStartE2EDuration="3.692862363s" totalImagesPullingTime="2.050207791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:34:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:34:32.689296604 +0000 UTC m=+600.653877684" watchObservedRunningTime="2026-07-30 19:34:32.692862363 +0000 UTC m=+600.657443444"
Jul 30 19:34:38.685232 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:38.685196    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-844cc9dc96-gs8n7"
Jul 30 19:34:38.712565 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:38.712523    2554 generic.go:356] "Generic (PLEG): container finished" podID="0582054c-2090-4cc6-9420-50011ca2d94f" containerID="56ebb206d02a6625bc0b7b7f7427790e59d4a6e40fb16dc699da18063a3c8679" exitCode=0
Jul 30 19:34:38.712747 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:38.712585    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v" event={"ID":"0582054c-2090-4cc6-9420-50011ca2d94f","Type":"ContainerDied","Data":"56ebb206d02a6625bc0b7b7f7427790e59d4a6e40fb16dc699da18063a3c8679"}
Jul 30 19:34:38.713079 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:38.713052    2554 scope.go:117] "RemoveContainer" containerID="56ebb206d02a6625bc0b7b7f7427790e59d4a6e40fb16dc699da18063a3c8679"
Jul 30 19:34:38.714224 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:38.714193    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 19:34:38.714338 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:38.714271    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 19:34:38.714338 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:38.714302    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 19:34:39.673326 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:39.673277    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:34:39.730380 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:39.730334    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v" event={"ID":"0582054c-2090-4cc6-9420-50011ca2d94f","Type":"ContainerStarted","Data":"3121a3ca25d2f1f83b8e70f4e21810f1c3f4598b14fc8095ac08649754205530"}
Jul 30 19:34:39.734598 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:39.734549    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:34:39.736183 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:39.736155    2554 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-fbm6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 30 19:34:39.736335 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:39.736218    2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v" podUID="0582054c-2090-4cc6-9420-50011ca2d94f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 30 19:34:39.801744 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:39.801711    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:34:40.734780 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:40.734742    2554 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-fbm6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 30 19:34:40.735306 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:40.734808    2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v" podUID="0582054c-2090-4cc6-9420-50011ca2d94f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 30 19:34:41.734824 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:41.734773    2554 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-fbm6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 30 19:34:41.735372 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:41.734856    2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v" podUID="0582054c-2090-4cc6-9420-50011ca2d94f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 30 19:34:42.133122 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:42.132203    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:34:42.735352 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:42.735312    2554 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-fbm6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 30 19:34:42.735795 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:42.735374    2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v" podUID="0582054c-2090-4cc6-9420-50011ca2d94f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 30 19:34:43.739049 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:43.739020    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:34:43.740136 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:43.740115    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fbm6v"
Jul 30 19:34:52.273940 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.273907    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x"]
Jul 30 19:34:52.277174 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.277152    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x"
Jul 30 19:34:52.280636 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.280605    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 30 19:34:52.280781 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.280670    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-78nfb\""
Jul 30 19:34:52.280781 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.280676    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 30 19:34:52.280781 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.280730    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 30 19:34:52.285728 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.285705    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3a5e6d2-c06f-4304-9910-fbf0ed329d09-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x\" (UID: \"c3a5e6d2-c06f-4304-9910-fbf0ed329d09\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x"
Jul 30 19:34:52.285848 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.285775    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3a5e6d2-c06f-4304-9910-fbf0ed329d09-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x\" (UID: \"c3a5e6d2-c06f-4304-9910-fbf0ed329d09\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x"
Jul 30 19:34:52.285848 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.285826    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3a5e6d2-c06f-4304-9910-fbf0ed329d09-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x\" (UID: \"c3a5e6d2-c06f-4304-9910-fbf0ed329d09\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x"
Jul 30 19:34:52.285985 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.285898    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3a5e6d2-c06f-4304-9910-fbf0ed329d09-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x\" (UID: \"c3a5e6d2-c06f-4304-9910-fbf0ed329d09\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x"
Jul 30 19:34:52.285985 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.285927    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3a5e6d2-c06f-4304-9910-fbf0ed329d09-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x\" (UID: \"c3a5e6d2-c06f-4304-9910-fbf0ed329d09\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x"
Jul 30 19:34:52.286180 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.286159    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4swh4\" (UniqueName: \"kubernetes.io/projected/c3a5e6d2-c06f-4304-9910-fbf0ed329d09-kube-api-access-4swh4\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x\" (UID: \"c3a5e6d2-c06f-4304-9910-fbf0ed329d09\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x"
Jul 30 19:34:52.289449 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.289430    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x"]
Jul 30 19:34:52.295163 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.295141    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:34:52.387029 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.386983    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3a5e6d2-c06f-4304-9910-fbf0ed329d09-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x\" (UID: \"c3a5e6d2-c06f-4304-9910-fbf0ed329d09\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x"
Jul 30 19:34:52.387210 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.387064    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3a5e6d2-c06f-4304-9910-fbf0ed329d09-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x\" (UID: \"c3a5e6d2-c06f-4304-9910-fbf0ed329d09\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x"
Jul 30 19:34:52.387210 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.387086    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3a5e6d2-c06f-4304-9910-fbf0ed329d09-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x\" (UID: \"c3a5e6d2-c06f-4304-9910-fbf0ed329d09\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x"
Jul 30 19:34:52.387210 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.387118    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4swh4\" (UniqueName: \"kubernetes.io/projected/c3a5e6d2-c06f-4304-9910-fbf0ed329d09-kube-api-access-4swh4\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x\" (UID: \"c3a5e6d2-c06f-4304-9910-fbf0ed329d09\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x"
Jul 30 19:34:52.387210 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.387152    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3a5e6d2-c06f-4304-9910-fbf0ed329d09-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x\" (UID: \"c3a5e6d2-c06f-4304-9910-fbf0ed329d09\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x"
Jul 30 19:34:52.387210 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.387191    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3a5e6d2-c06f-4304-9910-fbf0ed329d09-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x\" (UID: \"c3a5e6d2-c06f-4304-9910-fbf0ed329d09\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x"
Jul 30 19:34:52.387531 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.387503    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3a5e6d2-c06f-4304-9910-fbf0ed329d09-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x\" (UID: \"c3a5e6d2-c06f-4304-9910-fbf0ed329d09\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x"
Jul 30 19:34:52.387600 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.387582    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3a5e6d2-c06f-4304-9910-fbf0ed329d09-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x\" (UID: \"c3a5e6d2-c06f-4304-9910-fbf0ed329d09\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x"
Jul 30 19:34:52.387662 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.387615    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3a5e6d2-c06f-4304-9910-fbf0ed329d09-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x\" (UID: \"c3a5e6d2-c06f-4304-9910-fbf0ed329d09\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x"
Jul 30 19:34:52.389634 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.389609    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3a5e6d2-c06f-4304-9910-fbf0ed329d09-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x\" (UID: \"c3a5e6d2-c06f-4304-9910-fbf0ed329d09\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x"
Jul 30 19:34:52.389747 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.389669    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3a5e6d2-c06f-4304-9910-fbf0ed329d09-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x\" (UID: \"c3a5e6d2-c06f-4304-9910-fbf0ed329d09\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x"
Jul 30 19:34:52.395182 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.395148    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4swh4\" (UniqueName: \"kubernetes.io/projected/c3a5e6d2-c06f-4304-9910-fbf0ed329d09-kube-api-access-4swh4\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x\" (UID: \"c3a5e6d2-c06f-4304-9910-fbf0ed329d09\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x"
Jul 30 19:34:52.589403 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.589305    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x"
Jul 30 19:34:52.735858 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.735826    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x"]
Jul 30 19:34:52.737119 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:34:52.737081    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3a5e6d2_c06f_4304_9910_fbf0ed329d09.slice/crio-e4f78e8333856b4b771476b9f0fc8c364904b2813ec1a75c74e5482db48e30d6 WatchSource:0}: Error finding container e4f78e8333856b4b771476b9f0fc8c364904b2813ec1a75c74e5482db48e30d6: Status 404 returned error can't find the container with id e4f78e8333856b4b771476b9f0fc8c364904b2813ec1a75c74e5482db48e30d6
Jul 30 19:34:52.738592 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.738570    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 19:34:52.802628 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:52.802593    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x" event={"ID":"c3a5e6d2-c06f-4304-9910-fbf0ed329d09","Type":"ContainerStarted","Data":"e4f78e8333856b4b771476b9f0fc8c364904b2813ec1a75c74e5482db48e30d6"}
Jul 30 19:34:53.809448 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:53.809407    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x" event={"ID":"c3a5e6d2-c06f-4304-9910-fbf0ed329d09","Type":"ContainerStarted","Data":"2fefd24a211da7ae9c2b0ac9fd13ed11444f81b39714e85579c6bd94ac7a4f48"}
Jul 30 19:34:53.831314 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:34:53.831261    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x" podStartSLOduration=0.884577144 podStartE2EDuration="1.831241104s" totalImagesPullingTime="946.66396ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:34:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:34:53.828885212 +0000 UTC m=+621.793466295" watchObservedRunningTime="2026-07-30 19:34:53.831241104 +0000 UTC m=+621.795822185"
Jul 30 19:35:02.589520 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:02.589427    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x"
Jul 30 19:35:02.589520 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:02.589491    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x"
Jul 30 19:35:02.603702 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:02.603676    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x"
Jul 30 19:35:02.870458 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:02.870373    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x"
Jul 30 19:35:06.402203 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:06.402171    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:35:18.570598 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:18.570558    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c"]
Jul 30 19:35:18.574112 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:18.574086    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c"
Jul 30 19:35:18.576407 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:18.576387    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 30 19:35:18.582527 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:18.582503    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c"]
Jul 30 19:35:18.673552 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:18.673519    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:35:18.745712 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:18.745678    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b90b185-1f26-4137-bd15-017cc5947428-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4j95c\" (UID: \"0b90b185-1f26-4137-bd15-017cc5947428\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c"
Jul 30 19:35:18.745913 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:18.745721    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b90b185-1f26-4137-bd15-017cc5947428-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4j95c\" (UID: \"0b90b185-1f26-4137-bd15-017cc5947428\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c"
Jul 30 19:35:18.745913 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:18.745800    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5h64\" (UniqueName: \"kubernetes.io/projected/0b90b185-1f26-4137-bd15-017cc5947428-kube-api-access-s5h64\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4j95c\" (UID: \"0b90b185-1f26-4137-bd15-017cc5947428\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c"
Jul 30 19:35:18.746063 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:18.745945    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b90b185-1f26-4137-bd15-017cc5947428-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4j95c\" (UID: \"0b90b185-1f26-4137-bd15-017cc5947428\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c"
Jul 30 19:35:18.746063 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:18.746010    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b90b185-1f26-4137-bd15-017cc5947428-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4j95c\" (UID: \"0b90b185-1f26-4137-bd15-017cc5947428\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c"
Jul 30 19:35:18.746172 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:18.746095    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b90b185-1f26-4137-bd15-017cc5947428-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4j95c\" (UID: \"0b90b185-1f26-4137-bd15-017cc5947428\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c"
Jul 30 19:35:18.847681 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:18.847571    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b90b185-1f26-4137-bd15-017cc5947428-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4j95c\" (UID: \"0b90b185-1f26-4137-bd15-017cc5947428\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c"
Jul 30 19:35:18.847681 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:18.847647    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5h64\" (UniqueName: \"kubernetes.io/projected/0b90b185-1f26-4137-bd15-017cc5947428-kube-api-access-s5h64\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4j95c\" (UID: \"0b90b185-1f26-4137-bd15-017cc5947428\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c"
Jul 30 19:35:18.847923 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:18.847721    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b90b185-1f26-4137-bd15-017cc5947428-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4j95c\" (UID: \"0b90b185-1f26-4137-bd15-017cc5947428\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c"
Jul 30 19:35:18.847923 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:18.847755    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b90b185-1f26-4137-bd15-017cc5947428-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4j95c\" (UID: \"0b90b185-1f26-4137-bd15-017cc5947428\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c"
Jul 30 19:35:18.847923 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:18.847827    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b90b185-1f26-4137-bd15-017cc5947428-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4j95c\" (UID: \"0b90b185-1f26-4137-bd15-017cc5947428\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c"
Jul 30 19:35:18.847923 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:18.847866    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b90b185-1f26-4137-bd15-017cc5947428-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4j95c\" (UID: \"0b90b185-1f26-4137-bd15-017cc5947428\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c"
Jul 30 19:35:18.848176 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:18.848154    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b90b185-1f26-4137-bd15-017cc5947428-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4j95c\" (UID: \"0b90b185-1f26-4137-bd15-017cc5947428\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c"
Jul 30 19:35:18.848233 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:18.848202    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b90b185-1f26-4137-bd15-017cc5947428-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4j95c\" (UID: \"0b90b185-1f26-4137-bd15-017cc5947428\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c"
Jul 30 19:35:18.848284 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:18.848260    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b90b185-1f26-4137-bd15-017cc5947428-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4j95c\" (UID: \"0b90b185-1f26-4137-bd15-017cc5947428\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c"
Jul 30 19:35:18.850087 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:18.850062    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b90b185-1f26-4137-bd15-017cc5947428-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4j95c\" (UID: \"0b90b185-1f26-4137-bd15-017cc5947428\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c"
Jul 30 19:35:18.850267 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:18.850250    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b90b185-1f26-4137-bd15-017cc5947428-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4j95c\" (UID: \"0b90b185-1f26-4137-bd15-017cc5947428\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c"
Jul 30 19:35:18.856032 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:18.856012    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5h64\" (UniqueName: \"kubernetes.io/projected/0b90b185-1f26-4137-bd15-017cc5947428-kube-api-access-s5h64\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4j95c\" (UID: \"0b90b185-1f26-4137-bd15-017cc5947428\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c"
Jul 30 19:35:18.885475 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:18.885434    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c"
Jul 30 19:35:19.029873 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:19.029835    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c"]
Jul 30 19:35:19.031473 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:35:19.031446    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b90b185_1f26_4137_bd15_017cc5947428.slice/crio-7bb884af78ef2c8e92e5d86cf7e72f9c3527f52d3084319693a6ece43bf93911 WatchSource:0}: Error finding container 7bb884af78ef2c8e92e5d86cf7e72f9c3527f52d3084319693a6ece43bf93911: Status 404 returned error can't find the container with id 7bb884af78ef2c8e92e5d86cf7e72f9c3527f52d3084319693a6ece43bf93911
Jul 30 19:35:19.955581 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:19.955534    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c" event={"ID":"0b90b185-1f26-4137-bd15-017cc5947428","Type":"ContainerStarted","Data":"68502a3aa4e675dcadb50b83c7508be7e8740c0ed0459b7884272c4e4338fcbb"}
Jul 30 19:35:19.956080 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:19.955600    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c" event={"ID":"0b90b185-1f26-4137-bd15-017cc5947428","Type":"ContainerStarted","Data":"7bb884af78ef2c8e92e5d86cf7e72f9c3527f52d3084319693a6ece43bf93911"}
Jul 30 19:35:19.974277 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:19.974221    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c" podStartSLOduration=1.802478483 podStartE2EDuration="1.974203599s" totalImagesPullingTime="171.725116ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:35:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:35:19.971310518 +0000 UTC m=+647.935891613" watchObservedRunningTime="2026-07-30 19:35:19.974203599 +0000 UTC m=+647.938784680"
Jul 30 19:35:25.411583 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:25.411545    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:35:25.588610 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:25.588575    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:35:28.886324 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:28.886286    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c"
Jul 30 19:35:28.886796 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:28.886350    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c"
Jul 30 19:35:28.905813 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:28.905784    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c"
Jul 30 19:35:29.016986 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:29.016937    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4j95c"
Jul 30 19:35:30.146932 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:30.146899    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:35:30.203168 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:30.203135    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:35:34.582245 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:34.582207    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:35:39.778080 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:35:39.778042    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:36:14.344116 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:36:14.344079    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:36:58.854686 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:36:58.854642    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:36:58.966834 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:36:58.966798    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:36:59.999866 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:36:59.999816    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:37:00.091800 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:37:00.091766    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:37:26.582527 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:37:26.582491    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:37:26.683785 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:37:26.683750    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:37:33.140988 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:37:33.140910    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:37:46.189397 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:37:46.189361    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-785db54bd5-hjlpt"]
Jul 30 19:37:46.189831 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:37:46.189568    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-785db54bd5-hjlpt" podUID="c7c791b0-e5e2-4cd2-991a-312f43fce85a" containerName="manager" containerID="cri-o://2f1d1ef596760d7b487def8e816cdc4f5bcefb2548800fec1116b314bcba0721" gracePeriod=30
Jul 30 19:37:46.448424 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:37:46.448359    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-785db54bd5-hjlpt"
Jul 30 19:37:46.577504 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:37:46.577473    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmw82\" (UniqueName: \"kubernetes.io/projected/c7c791b0-e5e2-4cd2-991a-312f43fce85a-kube-api-access-lmw82\") pod \"c7c791b0-e5e2-4cd2-991a-312f43fce85a\" (UID: \"c7c791b0-e5e2-4cd2-991a-312f43fce85a\") "
Jul 30 19:37:46.577686 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:37:46.577577    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c7c791b0-e5e2-4cd2-991a-312f43fce85a-webhook-cert\") pod \"c7c791b0-e5e2-4cd2-991a-312f43fce85a\" (UID: \"c7c791b0-e5e2-4cd2-991a-312f43fce85a\") "
Jul 30 19:37:46.579708 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:37:46.579676    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7c791b0-e5e2-4cd2-991a-312f43fce85a-kube-api-access-lmw82" (OuterVolumeSpecName: "kube-api-access-lmw82") pod "c7c791b0-e5e2-4cd2-991a-312f43fce85a" (UID: "c7c791b0-e5e2-4cd2-991a-312f43fce85a"). InnerVolumeSpecName "kube-api-access-lmw82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:37:46.579708 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:37:46.579689    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7c791b0-e5e2-4cd2-991a-312f43fce85a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c7c791b0-e5e2-4cd2-991a-312f43fce85a" (UID: "c7c791b0-e5e2-4cd2-991a-312f43fce85a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 19:37:46.679106 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:37:46.679069    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmw82\" (UniqueName: \"kubernetes.io/projected/c7c791b0-e5e2-4cd2-991a-312f43fce85a-kube-api-access-lmw82\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:37:46.679106 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:37:46.679102    2554 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c7c791b0-e5e2-4cd2-991a-312f43fce85a-webhook-cert\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:37:46.787133 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:37:46.787095    2554 generic.go:356] "Generic (PLEG): container finished" podID="c7c791b0-e5e2-4cd2-991a-312f43fce85a" containerID="2f1d1ef596760d7b487def8e816cdc4f5bcefb2548800fec1116b314bcba0721" exitCode=0
Jul 30 19:37:46.787321 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:37:46.787144    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-785db54bd5-hjlpt"
Jul 30 19:37:46.787321 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:37:46.787154    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-785db54bd5-hjlpt" event={"ID":"c7c791b0-e5e2-4cd2-991a-312f43fce85a","Type":"ContainerDied","Data":"2f1d1ef596760d7b487def8e816cdc4f5bcefb2548800fec1116b314bcba0721"}
Jul 30 19:37:46.787321 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:37:46.787216    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-785db54bd5-hjlpt" event={"ID":"c7c791b0-e5e2-4cd2-991a-312f43fce85a","Type":"ContainerDied","Data":"1681ecfcd2c2e98ffa7d3e8a78aaf3058d22a7f5e2e96cdacfe672019788fbfc"}
Jul 30 19:37:46.787321 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:37:46.787244    2554 scope.go:117] "RemoveContainer" containerID="2f1d1ef596760d7b487def8e816cdc4f5bcefb2548800fec1116b314bcba0721"
Jul 30 19:37:46.797194 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:37:46.797175    2554 scope.go:117] "RemoveContainer" containerID="2f1d1ef596760d7b487def8e816cdc4f5bcefb2548800fec1116b314bcba0721"
Jul 30 19:37:46.797471 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:37:46.797445    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f1d1ef596760d7b487def8e816cdc4f5bcefb2548800fec1116b314bcba0721\": container with ID starting with 2f1d1ef596760d7b487def8e816cdc4f5bcefb2548800fec1116b314bcba0721 not found: ID does not exist" containerID="2f1d1ef596760d7b487def8e816cdc4f5bcefb2548800fec1116b314bcba0721"
Jul 30 19:37:46.797532 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:37:46.797485    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f1d1ef596760d7b487def8e816cdc4f5bcefb2548800fec1116b314bcba0721"} err="failed to get container status \"2f1d1ef596760d7b487def8e816cdc4f5bcefb2548800fec1116b314bcba0721\": rpc error: code = NotFound desc = could not find container \"2f1d1ef596760d7b487def8e816cdc4f5bcefb2548800fec1116b314bcba0721\": container with ID starting with 2f1d1ef596760d7b487def8e816cdc4f5bcefb2548800fec1116b314bcba0721 not found: ID does not exist"
Jul 30 19:37:46.809820 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:37:46.809796    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-785db54bd5-hjlpt"]
Jul 30 19:37:46.815325 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:37:46.815301    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-785db54bd5-hjlpt"]
Jul 30 19:37:48.492138 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:37:48.488034    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7c791b0-e5e2-4cd2-991a-312f43fce85a" path="/var/lib/kubelet/pods/c7c791b0-e5e2-4cd2-991a-312f43fce85a/volumes"
Jul 30 19:38:47.626162 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:38:47.626124    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-785db54bd5-p9fbx"]
Jul 30 19:38:47.626804 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:38:47.626555    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7c791b0-e5e2-4cd2-991a-312f43fce85a" containerName="manager"
Jul 30 19:38:47.626804 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:38:47.626567    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7c791b0-e5e2-4cd2-991a-312f43fce85a" containerName="manager"
Jul 30 19:38:47.626804 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:38:47.626634    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7c791b0-e5e2-4cd2-991a-312f43fce85a" containerName="manager"
Jul 30 19:38:47.628743 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:38:47.628722    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-785db54bd5-p9fbx"
Jul 30 19:38:47.632481 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:38:47.632457    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 30 19:38:47.632606 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:38:47.632479    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6n8w5\""
Jul 30 19:38:47.632606 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:38:47.632462    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 30 19:38:47.640491 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:38:47.640469    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-785db54bd5-p9fbx"]
Jul 30 19:38:47.664299 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:38:47.664250    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdxhs\" (UniqueName: \"kubernetes.io/projected/433cc9d0-16b9-442f-b45c-eaff53ec0c75-kube-api-access-hdxhs\") pod \"maas-controller-785db54bd5-p9fbx\" (UID: \"433cc9d0-16b9-442f-b45c-eaff53ec0c75\") " pod="opendatahub/maas-controller-785db54bd5-p9fbx"
Jul 30 19:38:47.664520 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:38:47.664333    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/433cc9d0-16b9-442f-b45c-eaff53ec0c75-webhook-cert\") pod \"maas-controller-785db54bd5-p9fbx\" (UID: \"433cc9d0-16b9-442f-b45c-eaff53ec0c75\") " pod="opendatahub/maas-controller-785db54bd5-p9fbx"
Jul 30 19:38:47.765536 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:38:47.765493    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdxhs\" (UniqueName: \"kubernetes.io/projected/433cc9d0-16b9-442f-b45c-eaff53ec0c75-kube-api-access-hdxhs\") pod \"maas-controller-785db54bd5-p9fbx\" (UID: \"433cc9d0-16b9-442f-b45c-eaff53ec0c75\") " pod="opendatahub/maas-controller-785db54bd5-p9fbx"
Jul 30 19:38:47.765719 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:38:47.765551    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/433cc9d0-16b9-442f-b45c-eaff53ec0c75-webhook-cert\") pod \"maas-controller-785db54bd5-p9fbx\" (UID: \"433cc9d0-16b9-442f-b45c-eaff53ec0c75\") " pod="opendatahub/maas-controller-785db54bd5-p9fbx"
Jul 30 19:38:47.768108 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:38:47.768075    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/433cc9d0-16b9-442f-b45c-eaff53ec0c75-webhook-cert\") pod \"maas-controller-785db54bd5-p9fbx\" (UID: \"433cc9d0-16b9-442f-b45c-eaff53ec0c75\") " pod="opendatahub/maas-controller-785db54bd5-p9fbx"
Jul 30 19:38:47.773453 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:38:47.773425    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdxhs\" (UniqueName: \"kubernetes.io/projected/433cc9d0-16b9-442f-b45c-eaff53ec0c75-kube-api-access-hdxhs\") pod \"maas-controller-785db54bd5-p9fbx\" (UID: \"433cc9d0-16b9-442f-b45c-eaff53ec0c75\") " pod="opendatahub/maas-controller-785db54bd5-p9fbx"
Jul 30 19:38:47.940797 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:38:47.940697    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-785db54bd5-p9fbx"
Jul 30 19:38:48.083421 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:38:48.083389    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-785db54bd5-p9fbx"]
Jul 30 19:38:48.084710 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:38:48.084685    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod433cc9d0_16b9_442f_b45c_eaff53ec0c75.slice/crio-b9a3f620e71fba66053db1b0d9801bf3415f684a38c033e0a96237a1018b2687 WatchSource:0}: Error finding container b9a3f620e71fba66053db1b0d9801bf3415f684a38c033e0a96237a1018b2687: Status 404 returned error can't find the container with id b9a3f620e71fba66053db1b0d9801bf3415f684a38c033e0a96237a1018b2687
Jul 30 19:38:48.137098 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:38:48.137059    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-785db54bd5-p9fbx" event={"ID":"433cc9d0-16b9-442f-b45c-eaff53ec0c75","Type":"ContainerStarted","Data":"b9a3f620e71fba66053db1b0d9801bf3415f684a38c033e0a96237a1018b2687"}
Jul 30 19:38:49.144005 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:38:49.143943    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-785db54bd5-p9fbx" event={"ID":"433cc9d0-16b9-442f-b45c-eaff53ec0c75","Type":"ContainerStarted","Data":"5e0844df5591bc03014712b81deb3a63110d5383a9e2a042c10dec41d2d5e777"}
Jul 30 19:38:49.144459 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:38:49.144025    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-785db54bd5-p9fbx"
Jul 30 19:38:49.162205 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:38:49.162153    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-785db54bd5-p9fbx" podStartSLOduration=1.672924938 podStartE2EDuration="2.162136448s" totalImagesPullingTime="489.21151ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:38:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:38:49.159638579 +0000 UTC m=+857.124219663" watchObservedRunningTime="2026-07-30 19:38:49.162136448 +0000 UTC m=+857.126717528"
Jul 30 19:39:00.152246 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:00.152213    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-785db54bd5-p9fbx"
Jul 30 19:39:04.727864 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:04.727816    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:39:04.798767 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:04.798735    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-785db54bd5-p9fbx"]
Jul 30 19:39:04.798972 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:04.798922    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-785db54bd5-p9fbx" podUID="433cc9d0-16b9-442f-b45c-eaff53ec0c75" containerName="manager" containerID="cri-o://5e0844df5591bc03014712b81deb3a63110d5383a9e2a042c10dec41d2d5e777" gracePeriod=30
Jul 30 19:39:05.049148 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:05.049123    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-785db54bd5-p9fbx"
Jul 30 19:39:05.142155 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:05.142113    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdxhs\" (UniqueName: \"kubernetes.io/projected/433cc9d0-16b9-442f-b45c-eaff53ec0c75-kube-api-access-hdxhs\") pod \"433cc9d0-16b9-442f-b45c-eaff53ec0c75\" (UID: \"433cc9d0-16b9-442f-b45c-eaff53ec0c75\") "
Jul 30 19:39:05.142336 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:05.142197    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/433cc9d0-16b9-442f-b45c-eaff53ec0c75-webhook-cert\") pod \"433cc9d0-16b9-442f-b45c-eaff53ec0c75\" (UID: \"433cc9d0-16b9-442f-b45c-eaff53ec0c75\") "
Jul 30 19:39:05.144191 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:05.144163    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/433cc9d0-16b9-442f-b45c-eaff53ec0c75-kube-api-access-hdxhs" (OuterVolumeSpecName: "kube-api-access-hdxhs") pod "433cc9d0-16b9-442f-b45c-eaff53ec0c75" (UID: "433cc9d0-16b9-442f-b45c-eaff53ec0c75"). InnerVolumeSpecName "kube-api-access-hdxhs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:39:05.144326 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:05.144207    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/433cc9d0-16b9-442f-b45c-eaff53ec0c75-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "433cc9d0-16b9-442f-b45c-eaff53ec0c75" (UID: "433cc9d0-16b9-442f-b45c-eaff53ec0c75"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 19:39:05.237618 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:05.237585    2554 generic.go:356] "Generic (PLEG): container finished" podID="433cc9d0-16b9-442f-b45c-eaff53ec0c75" containerID="5e0844df5591bc03014712b81deb3a63110d5383a9e2a042c10dec41d2d5e777" exitCode=0
Jul 30 19:39:05.237814 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:05.237632    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-785db54bd5-p9fbx"
Jul 30 19:39:05.237814 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:05.237676    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-785db54bd5-p9fbx" event={"ID":"433cc9d0-16b9-442f-b45c-eaff53ec0c75","Type":"ContainerDied","Data":"5e0844df5591bc03014712b81deb3a63110d5383a9e2a042c10dec41d2d5e777"}
Jul 30 19:39:05.237814 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:05.237728    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-785db54bd5-p9fbx" event={"ID":"433cc9d0-16b9-442f-b45c-eaff53ec0c75","Type":"ContainerDied","Data":"b9a3f620e71fba66053db1b0d9801bf3415f684a38c033e0a96237a1018b2687"}
Jul 30 19:39:05.237814 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:05.237752    2554 scope.go:117] "RemoveContainer" containerID="5e0844df5591bc03014712b81deb3a63110d5383a9e2a042c10dec41d2d5e777"
Jul 30 19:39:05.242946 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:05.242922    2554 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/433cc9d0-16b9-442f-b45c-eaff53ec0c75-webhook-cert\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:39:05.243076 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:05.242944    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdxhs\" (UniqueName: \"kubernetes.io/projected/433cc9d0-16b9-442f-b45c-eaff53ec0c75-kube-api-access-hdxhs\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:39:05.247783 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:05.247760    2554 scope.go:117] "RemoveContainer" containerID="5e0844df5591bc03014712b81deb3a63110d5383a9e2a042c10dec41d2d5e777"
Jul 30 19:39:05.248087 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:39:05.248069    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e0844df5591bc03014712b81deb3a63110d5383a9e2a042c10dec41d2d5e777\": container with ID starting with 5e0844df5591bc03014712b81deb3a63110d5383a9e2a042c10dec41d2d5e777 not found: ID does not exist" containerID="5e0844df5591bc03014712b81deb3a63110d5383a9e2a042c10dec41d2d5e777"
Jul 30 19:39:05.248170 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:05.248095    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e0844df5591bc03014712b81deb3a63110d5383a9e2a042c10dec41d2d5e777"} err="failed to get container status \"5e0844df5591bc03014712b81deb3a63110d5383a9e2a042c10dec41d2d5e777\": rpc error: code = NotFound desc = could not find container \"5e0844df5591bc03014712b81deb3a63110d5383a9e2a042c10dec41d2d5e777\": container with ID starting with 5e0844df5591bc03014712b81deb3a63110d5383a9e2a042c10dec41d2d5e777 not found: ID does not exist"
Jul 30 19:39:05.260348 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:05.260320    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-785db54bd5-p9fbx"]
Jul 30 19:39:05.265042 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:05.265015    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-785db54bd5-p9fbx"]
Jul 30 19:39:05.930481 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:05.930443    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:39:06.046045 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:06.046008    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-785db54bd5-c59kr"]
Jul 30 19:39:06.046552 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:06.046537    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="433cc9d0-16b9-442f-b45c-eaff53ec0c75" containerName="manager"
Jul 30 19:39:06.046609 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:06.046554    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="433cc9d0-16b9-442f-b45c-eaff53ec0c75" containerName="manager"
Jul 30 19:39:06.046711 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:06.046700    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="433cc9d0-16b9-442f-b45c-eaff53ec0c75" containerName="manager"
Jul 30 19:39:06.049832 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:06.049807    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-785db54bd5-c59kr"
Jul 30 19:39:06.053257 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:06.053231    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6n8w5\""
Jul 30 19:39:06.053430 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:06.053231    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 30 19:39:06.053430 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:06.053231    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 30 19:39:06.058446 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:06.058427    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-785db54bd5-c59kr"]
Jul 30 19:39:06.150982 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:06.150928    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f9e40da3-e49e-4085-8d78-749bc2f9394c-webhook-cert\") pod \"maas-controller-785db54bd5-c59kr\" (UID: \"f9e40da3-e49e-4085-8d78-749bc2f9394c\") " pod="opendatahub/maas-controller-785db54bd5-c59kr"
Jul 30 19:39:06.151154 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:06.151043    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzfhl\" (UniqueName: \"kubernetes.io/projected/f9e40da3-e49e-4085-8d78-749bc2f9394c-kube-api-access-tzfhl\") pod \"maas-controller-785db54bd5-c59kr\" (UID: \"f9e40da3-e49e-4085-8d78-749bc2f9394c\") " pod="opendatahub/maas-controller-785db54bd5-c59kr"
Jul 30 19:39:06.252514 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:06.252484    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f9e40da3-e49e-4085-8d78-749bc2f9394c-webhook-cert\") pod \"maas-controller-785db54bd5-c59kr\" (UID: \"f9e40da3-e49e-4085-8d78-749bc2f9394c\") " pod="opendatahub/maas-controller-785db54bd5-c59kr"
Jul 30 19:39:06.252710 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:06.252554    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzfhl\" (UniqueName: \"kubernetes.io/projected/f9e40da3-e49e-4085-8d78-749bc2f9394c-kube-api-access-tzfhl\") pod \"maas-controller-785db54bd5-c59kr\" (UID: \"f9e40da3-e49e-4085-8d78-749bc2f9394c\") " pod="opendatahub/maas-controller-785db54bd5-c59kr"
Jul 30 19:39:06.254961 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:06.254919    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f9e40da3-e49e-4085-8d78-749bc2f9394c-webhook-cert\") pod \"maas-controller-785db54bd5-c59kr\" (UID: \"f9e40da3-e49e-4085-8d78-749bc2f9394c\") " pod="opendatahub/maas-controller-785db54bd5-c59kr"
Jul 30 19:39:06.263649 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:06.263619    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzfhl\" (UniqueName: \"kubernetes.io/projected/f9e40da3-e49e-4085-8d78-749bc2f9394c-kube-api-access-tzfhl\") pod \"maas-controller-785db54bd5-c59kr\" (UID: \"f9e40da3-e49e-4085-8d78-749bc2f9394c\") " pod="opendatahub/maas-controller-785db54bd5-c59kr"
Jul 30 19:39:06.361817 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:06.361767    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-785db54bd5-c59kr"
Jul 30 19:39:06.488628 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:06.488591    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="433cc9d0-16b9-442f-b45c-eaff53ec0c75" path="/var/lib/kubelet/pods/433cc9d0-16b9-442f-b45c-eaff53ec0c75/volumes"
Jul 30 19:39:06.503053 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:06.503001    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-785db54bd5-c59kr"]
Jul 30 19:39:06.504843 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:39:06.504817    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9e40da3_e49e_4085_8d78_749bc2f9394c.slice/crio-f3561fb3dfde1ebdb0e98d80c69a60479bf05cb9b86f531b478f818177c88b0d WatchSource:0}: Error finding container f3561fb3dfde1ebdb0e98d80c69a60479bf05cb9b86f531b478f818177c88b0d: Status 404 returned error can't find the container with id f3561fb3dfde1ebdb0e98d80c69a60479bf05cb9b86f531b478f818177c88b0d
Jul 30 19:39:07.253062 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:07.253021    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-785db54bd5-c59kr" event={"ID":"f9e40da3-e49e-4085-8d78-749bc2f9394c","Type":"ContainerStarted","Data":"f12cdcaad35357a3e47b965dcce12838d1e8517a93b5a45b13320e7598f56f3b"}
Jul 30 19:39:07.253502 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:07.253086    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-785db54bd5-c59kr" event={"ID":"f9e40da3-e49e-4085-8d78-749bc2f9394c","Type":"ContainerStarted","Data":"f3561fb3dfde1ebdb0e98d80c69a60479bf05cb9b86f531b478f818177c88b0d"}
Jul 30 19:39:07.253502 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:07.253117    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-785db54bd5-c59kr"
Jul 30 19:39:07.270473 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:07.270424    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-785db54bd5-c59kr" podStartSLOduration=0.88381304 podStartE2EDuration="1.27040654s" totalImagesPullingTime="386.5935ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:39:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:39:07.27014061 +0000 UTC m=+875.234721692" watchObservedRunningTime="2026-07-30 19:39:07.27040654 +0000 UTC m=+875.234987619"
Jul 30 19:39:18.263811 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:18.263774    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-785db54bd5-c59kr"
Jul 30 19:39:18.267766 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:18.267738    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:39:21.489840 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:21.489801    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:39:32.492792 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:32.492757    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/1.log"
Jul 30 19:39:32.494068 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:32.494034    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/1.log"
Jul 30 19:39:32.498428 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:32.498406    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 19:39:32.499649 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:39:32.499627    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 19:40:01.993553 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:40:01.993517    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:40:04.571731 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:40:04.571695    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:40:07.595460 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:40:07.595422    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:40:07.696285 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:40:07.696253    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:40:07.794444 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:40:07.794413    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:40:07.897414 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:40:07.897330    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:40:13.607964 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:40:13.607929    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:40:13.708104 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:40:13.708056    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:42:34.925430 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:42:34.925337    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:42:34.979759 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:42:34.979723    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:44:32.535141 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:44:32.535050    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/1.log"
Jul 30 19:44:32.539860 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:44:32.539830    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 19:44:32.540058 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:44:32.540015    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/1.log"
Jul 30 19:44:32.544652 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:44:32.544632    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 19:44:34.569637 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:44:34.569598    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:44:34.675183 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:44:34.675154    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:45:00.148813 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:00.148774    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757345-v7wzk"]
Jul 30 19:45:00.152546 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:00.152522    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757345-v7wzk"
Jul 30 19:45:00.155126 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:00.155104    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7t7kk\""
Jul 30 19:45:00.162524 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:00.162505    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757345-v7wzk"
Jul 30 19:45:00.169915 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:00.169887    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757345-v7wzk"]
Jul 30 19:45:00.311082 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:00.311048    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757345-v7wzk"]
Jul 30 19:45:00.311555 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:45:00.311531    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89254ead_4e44_461c_a98e_31d90cf9d972.slice/crio-235a4e007faf4d6469f110a4b7e3d3397b597172dc233f2e1cb23ca3552aa59a WatchSource:0}: Error finding container 235a4e007faf4d6469f110a4b7e3d3397b597172dc233f2e1cb23ca3552aa59a: Status 404 returned error can't find the container with id 235a4e007faf4d6469f110a4b7e3d3397b597172dc233f2e1cb23ca3552aa59a
Jul 30 19:45:00.313680 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:00.313662    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 19:45:01.293925 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:01.293883    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757345-v7wzk" event={"ID":"89254ead-4e44-461c-a98e-31d90cf9d972","Type":"ContainerStarted","Data":"235a4e007faf4d6469f110a4b7e3d3397b597172dc233f2e1cb23ca3552aa59a"}
Jul 30 19:45:03.307745 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:03.307643    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757345-v7wzk" event={"ID":"89254ead-4e44-461c-a98e-31d90cf9d972","Type":"ContainerStarted","Data":"8785336b5e810ca6ec4a6719e1138ee90a0f5f1303c3ba665dd0e842c8b34112"}
Jul 30 19:45:03.323125 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:03.323052    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757345-v7wzk" podStartSLOduration=0.91300602 podStartE2EDuration="3.323029226s" totalImagesPullingTime="2.410023206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:45:03.322453101 +0000 UTC m=+1231.287034189" watchObservedRunningTime="2026-07-30 19:45:03.323029226 +0000 UTC m=+1231.287610310"
Jul 30 19:45:04.314798 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:04.314759    2554 generic.go:356] "Generic (PLEG): container finished" podID="89254ead-4e44-461c-a98e-31d90cf9d972" containerID="8785336b5e810ca6ec4a6719e1138ee90a0f5f1303c3ba665dd0e842c8b34112" exitCode=0
Jul 30 19:45:04.315207 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:04.314818    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757345-v7wzk" event={"ID":"89254ead-4e44-461c-a98e-31d90cf9d972","Type":"ContainerDied","Data":"8785336b5e810ca6ec4a6719e1138ee90a0f5f1303c3ba665dd0e842c8b34112"}
Jul 30 19:45:05.457795 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:05.457766    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757345-v7wzk"
Jul 30 19:45:06.328626 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:06.328594    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757345-v7wzk"
Jul 30 19:45:06.328828 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:06.328635    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757345-v7wzk" event={"ID":"89254ead-4e44-461c-a98e-31d90cf9d972","Type":"ContainerDied","Data":"235a4e007faf4d6469f110a4b7e3d3397b597172dc233f2e1cb23ca3552aa59a"}
Jul 30 19:45:06.328828 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:06.328695    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="235a4e007faf4d6469f110a4b7e3d3397b597172dc233f2e1cb23ca3552aa59a"
Jul 30 19:45:51.791870 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:51.791828    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lzkrl"]
Jul 30 19:45:51.792420 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:51.792078    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lzkrl" podUID="d4b8740b-ed13-4da7-84cc-b60b33f80e54" containerName="manager" containerID="cri-o://b9cd3a7bd7f73d290afff4ea03b683823409a884fc1eff7c01fc5ea808ed2ef1" gracePeriod=10
Jul 30 19:45:52.051006 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:52.050927    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lzkrl"
Jul 30 19:45:52.188602 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:52.188563    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhg2q\" (UniqueName: \"kubernetes.io/projected/d4b8740b-ed13-4da7-84cc-b60b33f80e54-kube-api-access-mhg2q\") pod \"d4b8740b-ed13-4da7-84cc-b60b33f80e54\" (UID: \"d4b8740b-ed13-4da7-84cc-b60b33f80e54\") "
Jul 30 19:45:52.188819 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:52.188625    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4b8740b-ed13-4da7-84cc-b60b33f80e54-extensions-socket-volume\") pod \"d4b8740b-ed13-4da7-84cc-b60b33f80e54\" (UID: \"d4b8740b-ed13-4da7-84cc-b60b33f80e54\") "
Jul 30 19:45:52.189078 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:52.189050    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4b8740b-ed13-4da7-84cc-b60b33f80e54-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d4b8740b-ed13-4da7-84cc-b60b33f80e54" (UID: "d4b8740b-ed13-4da7-84cc-b60b33f80e54"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:45:52.190754 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:52.190732    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4b8740b-ed13-4da7-84cc-b60b33f80e54-kube-api-access-mhg2q" (OuterVolumeSpecName: "kube-api-access-mhg2q") pod "d4b8740b-ed13-4da7-84cc-b60b33f80e54" (UID: "d4b8740b-ed13-4da7-84cc-b60b33f80e54"). InnerVolumeSpecName "kube-api-access-mhg2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:45:52.289515 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:52.289478    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhg2q\" (UniqueName: \"kubernetes.io/projected/d4b8740b-ed13-4da7-84cc-b60b33f80e54-kube-api-access-mhg2q\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:45:52.289515 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:52.289510    2554 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4b8740b-ed13-4da7-84cc-b60b33f80e54-extensions-socket-volume\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:45:52.597203 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:52.597164    2554 generic.go:356] "Generic (PLEG): container finished" podID="d4b8740b-ed13-4da7-84cc-b60b33f80e54" containerID="b9cd3a7bd7f73d290afff4ea03b683823409a884fc1eff7c01fc5ea808ed2ef1" exitCode=0
Jul 30 19:45:52.597203 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:52.597201    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lzkrl"
Jul 30 19:45:52.597473 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:52.597235    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lzkrl" event={"ID":"d4b8740b-ed13-4da7-84cc-b60b33f80e54","Type":"ContainerDied","Data":"b9cd3a7bd7f73d290afff4ea03b683823409a884fc1eff7c01fc5ea808ed2ef1"}
Jul 30 19:45:52.597473 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:52.597295    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lzkrl" event={"ID":"d4b8740b-ed13-4da7-84cc-b60b33f80e54","Type":"ContainerDied","Data":"3512850fc9492164481bb504b0e779b4fafaeae49cf8f8ecd78db57c81b84e1c"}
Jul 30 19:45:52.597473 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:52.597324    2554 scope.go:117] "RemoveContainer" containerID="b9cd3a7bd7f73d290afff4ea03b683823409a884fc1eff7c01fc5ea808ed2ef1"
Jul 30 19:45:52.608046 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:52.608025    2554 scope.go:117] "RemoveContainer" containerID="b9cd3a7bd7f73d290afff4ea03b683823409a884fc1eff7c01fc5ea808ed2ef1"
Jul 30 19:45:52.608362 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:45:52.608343    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9cd3a7bd7f73d290afff4ea03b683823409a884fc1eff7c01fc5ea808ed2ef1\": container with ID starting with b9cd3a7bd7f73d290afff4ea03b683823409a884fc1eff7c01fc5ea808ed2ef1 not found: ID does not exist" containerID="b9cd3a7bd7f73d290afff4ea03b683823409a884fc1eff7c01fc5ea808ed2ef1"
Jul 30 19:45:52.608408 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:52.608372    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9cd3a7bd7f73d290afff4ea03b683823409a884fc1eff7c01fc5ea808ed2ef1"} err="failed to get container status \"b9cd3a7bd7f73d290afff4ea03b683823409a884fc1eff7c01fc5ea808ed2ef1\": rpc error: code = NotFound desc = could not find container \"b9cd3a7bd7f73d290afff4ea03b683823409a884fc1eff7c01fc5ea808ed2ef1\": container with ID starting with b9cd3a7bd7f73d290afff4ea03b683823409a884fc1eff7c01fc5ea808ed2ef1 not found: ID does not exist"
Jul 30 19:45:52.616442 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:52.616414    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lzkrl"]
Jul 30 19:45:52.619717 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:52.619689    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lzkrl"]
Jul 30 19:45:53.265635 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:53.265601    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:45:54.487568 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:45:54.487522    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4b8740b-ed13-4da7-84cc-b60b33f80e54" path="/var/lib/kubelet/pods/d4b8740b-ed13-4da7-84cc-b60b33f80e54/volumes"
Jul 30 19:46:35.492560 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:46:35.492509    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:46:52.476857 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:46:52.476816    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjn86"]
Jul 30 19:46:52.477276 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:46:52.477236    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4b8740b-ed13-4da7-84cc-b60b33f80e54" containerName="manager"
Jul 30 19:46:52.477276 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:46:52.477248    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4b8740b-ed13-4da7-84cc-b60b33f80e54" containerName="manager"
Jul 30 19:46:52.477276 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:46:52.477265    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89254ead-4e44-461c-a98e-31d90cf9d972" containerName="cleanup"
Jul 30 19:46:52.477276 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:46:52.477271    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="89254ead-4e44-461c-a98e-31d90cf9d972" containerName="cleanup"
Jul 30 19:46:52.477410 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:46:52.477348    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4b8740b-ed13-4da7-84cc-b60b33f80e54" containerName="manager"
Jul 30 19:46:52.477410 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:46:52.477358    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="89254ead-4e44-461c-a98e-31d90cf9d972" containerName="cleanup"
Jul 30 19:46:52.480237 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:46:52.480214    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjn86"
Jul 30 19:46:52.485727 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:46:52.485703    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mpd9r\""
Jul 30 19:46:52.496072 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:46:52.496051    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjn86"]
Jul 30 19:46:52.561349 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:46:52.561300    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkbsn\" (UniqueName: \"kubernetes.io/projected/9ec1b2c8-32ad-4372-ac57-06dec9fb6077-kube-api-access-tkbsn\") pod \"kuadrant-operator-controller-manager-86878c769f-bjn86\" (UID: \"9ec1b2c8-32ad-4372-ac57-06dec9fb6077\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjn86"
Jul 30 19:46:52.561548 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:46:52.561377    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9ec1b2c8-32ad-4372-ac57-06dec9fb6077-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bjn86\" (UID: \"9ec1b2c8-32ad-4372-ac57-06dec9fb6077\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjn86"
Jul 30 19:46:52.662145 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:46:52.662108    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkbsn\" (UniqueName: \"kubernetes.io/projected/9ec1b2c8-32ad-4372-ac57-06dec9fb6077-kube-api-access-tkbsn\") pod \"kuadrant-operator-controller-manager-86878c769f-bjn86\" (UID: \"9ec1b2c8-32ad-4372-ac57-06dec9fb6077\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjn86"
Jul 30 19:46:52.662316 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:46:52.662205    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9ec1b2c8-32ad-4372-ac57-06dec9fb6077-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bjn86\" (UID: \"9ec1b2c8-32ad-4372-ac57-06dec9fb6077\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjn86"
Jul 30 19:46:52.662625 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:46:52.662603    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9ec1b2c8-32ad-4372-ac57-06dec9fb6077-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bjn86\" (UID: \"9ec1b2c8-32ad-4372-ac57-06dec9fb6077\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjn86"
Jul 30 19:46:52.675705 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:46:52.675662    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkbsn\" (UniqueName: \"kubernetes.io/projected/9ec1b2c8-32ad-4372-ac57-06dec9fb6077-kube-api-access-tkbsn\") pod \"kuadrant-operator-controller-manager-86878c769f-bjn86\" (UID: \"9ec1b2c8-32ad-4372-ac57-06dec9fb6077\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjn86"
Jul 30 19:46:52.793575 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:46:52.793473    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjn86"
Jul 30 19:46:52.943922 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:46:52.943892    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjn86"]
Jul 30 19:46:52.947902 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:46:52.947874    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ec1b2c8_32ad_4372_ac57_06dec9fb6077.slice/crio-c11abc6b5b4f1d27a9393193fdcc4182e81f6cb611cbe47422af4b542d5123e3 WatchSource:0}: Error finding container c11abc6b5b4f1d27a9393193fdcc4182e81f6cb611cbe47422af4b542d5123e3: Status 404 returned error can't find the container with id c11abc6b5b4f1d27a9393193fdcc4182e81f6cb611cbe47422af4b542d5123e3
Jul 30 19:46:53.947392 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:46:53.947353    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjn86" event={"ID":"9ec1b2c8-32ad-4372-ac57-06dec9fb6077","Type":"ContainerStarted","Data":"e45e3e5a7738ecdfce2cb55420a586b9961148b09075175cb95ed8de9d64f025"}
Jul 30 19:46:53.947829 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:46:53.947419    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjn86"
Jul 30 19:46:53.947829 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:46:53.947438    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjn86" event={"ID":"9ec1b2c8-32ad-4372-ac57-06dec9fb6077","Type":"ContainerStarted","Data":"c11abc6b5b4f1d27a9393193fdcc4182e81f6cb611cbe47422af4b542d5123e3"}
Jul 30 19:46:53.983285 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:46:53.983227    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjn86" podStartSLOduration=1.983209703 podStartE2EDuration="1.983209703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:46:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:46:53.97923419 +0000 UTC m=+1341.943815271" watchObservedRunningTime="2026-07-30 19:46:53.983209703 +0000 UTC m=+1341.947790784"
Jul 30 19:47:04.955279 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:47:04.955192    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjn86"
Jul 30 19:49:32.579766 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:49:32.579737    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/1.log"
Jul 30 19:49:32.584621 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:49:32.584597    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 19:49:32.586834 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:49:32.586811    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/1.log"
Jul 30 19:49:32.591470 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:49:32.591452    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 19:50:14.101456 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:50:14.101396    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:50:14.152972 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:50:14.152921    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:50:22.596519 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:50:22.596475    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:50:33.299415 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:50:33.299373    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:51:22.168651 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:51:22.168606    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:51:25.221732 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:51:25.221629    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:52:05.250808 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:52:05.250776    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:53:54.143183 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:53:54.143146    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:53:54.231199 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:53:54.231166    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:54:32.622326 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:54:32.622295    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/1.log"
Jul 30 19:54:32.627268 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:54:32.627240    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 19:54:32.629875 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:54:32.629856    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/1.log"
Jul 30 19:54:32.634364 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:54:32.634348    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 19:55:22.754848 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:55:22.754807    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:55:59.269806 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:55:59.269718    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:56:02.339032 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:02.338997    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:56:02.436269 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:02.436237    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:56:02.788998 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:02.788931    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:56:08.405860 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:08.405820    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:56:08.461788 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:08.461757    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:56:08.649880 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:08.649846    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:56:08.914796 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:08.914748    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"]
Jul 30 19:56:08.917577 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:08.917554    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:08.919916 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:08.919890    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-6d3e9f39-openshift-default-dockercfg-gxw69\""
Jul 30 19:56:08.925655 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:08.925634    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:56:08.938460 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:08.938432    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"]
Jul 30 19:56:09.058216 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.058179    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-workload-socket\") pod \"e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") " pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:09.058413 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.058233    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-istio-envoy\") pod \"e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") " pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:09.058413 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.058279    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-credential-socket\") pod \"e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") " pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:09.058413 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.058294    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-workload-certs\") pod \"e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") " pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:09.058413 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.058310    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-istio-data\") pod \"e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") " pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:09.058413 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.058381    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c514fd5e-706e-453f-859a-31c6d8609468-istio-token\") pod \"e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") " pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:09.058619 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.058423    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c514fd5e-706e-453f-859a-31c6d8609468-istio-podinfo\") pod \"e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") " pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:09.058619 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.058473    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74s5d\" (UniqueName: \"kubernetes.io/projected/c514fd5e-706e-453f-859a-31c6d8609468-kube-api-access-74s5d\") pod \"e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") " pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:09.058619 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.058507    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c514fd5e-706e-453f-859a-31c6d8609468-istiod-ca-cert\") pod \"e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") " pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:09.159164 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.159125    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-workload-socket\") pod \"e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") " pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:09.159371 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.159192    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-istio-envoy\") pod \"e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") " pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:09.159371 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.159231    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-credential-socket\") pod \"e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") " pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:09.159371 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.159254    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-workload-certs\") pod \"e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") " pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:09.159371 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.159280    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-istio-data\") pod \"e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") " pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:09.159371 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.159326    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c514fd5e-706e-453f-859a-31c6d8609468-istio-token\") pod \"e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") " pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:09.159371 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.159363    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c514fd5e-706e-453f-859a-31c6d8609468-istio-podinfo\") pod \"e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") " pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:09.159715 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.159394    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74s5d\" (UniqueName: \"kubernetes.io/projected/c514fd5e-706e-453f-859a-31c6d8609468-kube-api-access-74s5d\") pod \"e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") " pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:09.159715 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.159436    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c514fd5e-706e-453f-859a-31c6d8609468-istiod-ca-cert\") pod \"e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") " pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:09.159715 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.159571    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-workload-socket\") pod \"e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") " pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:09.159715 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.159650    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-credential-socket\") pod \"e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") " pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:09.159715 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.159706    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-workload-certs\") pod \"e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") " pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:09.160051 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.159750    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-istio-data\") pod \"e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") " pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:09.160254 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.160232    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c514fd5e-706e-453f-859a-31c6d8609468-istiod-ca-cert\") pod \"e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") " pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:09.161590 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.161565    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-istio-envoy\") pod \"e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") " pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:09.162044 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.162026    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c514fd5e-706e-453f-859a-31c6d8609468-istio-podinfo\") pod \"e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") " pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:09.167672 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.167614    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c514fd5e-706e-453f-859a-31c6d8609468-istio-token\") pod \"e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") " pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:09.167767 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.167701    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74s5d\" (UniqueName: \"kubernetes.io/projected/c514fd5e-706e-453f-859a-31c6d8609468-kube-api-access-74s5d\") pod \"e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") " pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:09.228444 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.228408    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:09.370092 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.370055    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"]
Jul 30 19:56:09.371552 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:56:09.371524    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc514fd5e_706e_453f_859a_31c6d8609468.slice/crio-192c5d43e44757ff40d6477ca4eab72d5980f9e816cae6a2a70671e3dcecd968 WatchSource:0}: Error finding container 192c5d43e44757ff40d6477ca4eab72d5980f9e816cae6a2a70671e3dcecd968: Status 404 returned error can't find the container with id 192c5d43e44757ff40d6477ca4eab72d5980f9e816cae6a2a70671e3dcecd968
Jul 30 19:56:09.373745 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.373716    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 19:56:09.373808 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.373786    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 19:56:09.373847 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.373817    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 19:56:09.452856 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:09.452822    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:56:10.158590 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:10.158555    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l" event={"ID":"c514fd5e-706e-453f-859a-31c6d8609468","Type":"ContainerStarted","Data":"41b243870a0f58263b3a518773801e367859b2c6eb1021873fba2ebac13e2bc1"}
Jul 30 19:56:10.158825 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:10.158635    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l" event={"ID":"c514fd5e-706e-453f-859a-31c6d8609468","Type":"ContainerStarted","Data":"192c5d43e44757ff40d6477ca4eab72d5980f9e816cae6a2a70671e3dcecd968"}
Jul 30 19:56:10.179530 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:10.179476    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l" podStartSLOduration=2.179461048 podStartE2EDuration="2.179461048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:56:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:56:10.178436874 +0000 UTC m=+1898.143017955" watchObservedRunningTime="2026-07-30 19:56:10.179461048 +0000 UTC m=+1898.144042130"
Jul 30 19:56:10.229337 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:10.229297    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:10.234573 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:10.234538    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:11.164378 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:11.164349    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:11.165282 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:11.165264    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:11.505578 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:11.505544    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:56:11.604248 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:11.604213    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:56:12.603909 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:12.603874    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk"]
Jul 30 19:56:12.606184 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:12.606168    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk"
Jul 30 19:56:12.608408 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:12.608388    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-6d3e9f39-serving-cert\""
Jul 30 19:56:12.618575 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:12.618550    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk"]
Jul 30 19:56:12.672935 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:12.672907    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:56:12.792354 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:12.792320    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/828993bd-2909-479c-98a4-fe1961df9c4a-maas-api-tls\") pod \"maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk\" (UID: \"828993bd-2909-479c-98a4-fe1961df9c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk"
Jul 30 19:56:12.792536 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:12.792408    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/828993bd-2909-479c-98a4-fe1961df9c4a-openshift-service-ca\") pod \"maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk\" (UID: \"828993bd-2909-479c-98a4-fe1961df9c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk"
Jul 30 19:56:12.792536 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:12.792431    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw8v7\" (UniqueName: \"kubernetes.io/projected/828993bd-2909-479c-98a4-fe1961df9c4a-kube-api-access-hw8v7\") pod \"maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk\" (UID: \"828993bd-2909-479c-98a4-fe1961df9c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk"
Jul 30 19:56:12.846349 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:12.846310    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:56:12.893920 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:12.893824    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/828993bd-2909-479c-98a4-fe1961df9c4a-maas-api-tls\") pod \"maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk\" (UID: \"828993bd-2909-479c-98a4-fe1961df9c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk"
Jul 30 19:56:12.894120 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:12.893928    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/828993bd-2909-479c-98a4-fe1961df9c4a-openshift-service-ca\") pod \"maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk\" (UID: \"828993bd-2909-479c-98a4-fe1961df9c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk"
Jul 30 19:56:12.894120 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:12.893969    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw8v7\" (UniqueName: \"kubernetes.io/projected/828993bd-2909-479c-98a4-fe1961df9c4a-kube-api-access-hw8v7\") pod \"maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk\" (UID: \"828993bd-2909-479c-98a4-fe1961df9c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk"
Jul 30 19:56:12.894680 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:12.894656    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/828993bd-2909-479c-98a4-fe1961df9c4a-openshift-service-ca\") pod \"maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk\" (UID: \"828993bd-2909-479c-98a4-fe1961df9c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk"
Jul 30 19:56:12.896333 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:12.896307    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/828993bd-2909-479c-98a4-fe1961df9c4a-maas-api-tls\") pod \"maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk\" (UID: \"828993bd-2909-479c-98a4-fe1961df9c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk"
Jul 30 19:56:12.901205 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:12.901178    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw8v7\" (UniqueName: \"kubernetes.io/projected/828993bd-2909-479c-98a4-fe1961df9c4a-kube-api-access-hw8v7\") pod \"maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk\" (UID: \"828993bd-2909-479c-98a4-fe1961df9c4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk"
Jul 30 19:56:12.920105 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:12.920079    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk"
Jul 30 19:56:13.071935 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:13.071906    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk"]
Jul 30 19:56:13.073532 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:56:13.073501    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod828993bd_2909_479c_98a4_fe1961df9c4a.slice/crio-956e0a1fef7ed953025421768efd855742a20766e7b9e0c35fc92a71d2a0142f WatchSource:0}: Error finding container 956e0a1fef7ed953025421768efd855742a20766e7b9e0c35fc92a71d2a0142f: Status 404 returned error can't find the container with id 956e0a1fef7ed953025421768efd855742a20766e7b9e0c35fc92a71d2a0142f
Jul 30 19:56:13.074941 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:13.074926    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 19:56:13.177577 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:13.177479    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk" event={"ID":"828993bd-2909-479c-98a4-fe1961df9c4a","Type":"ContainerStarted","Data":"956e0a1fef7ed953025421768efd855742a20766e7b9e0c35fc92a71d2a0142f"}
Jul 30 19:56:13.191794 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:13.191763    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:56:13.287123 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:13.287092    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:56:14.185750 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:14.185709    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk" event={"ID":"828993bd-2909-479c-98a4-fe1961df9c4a","Type":"ContainerStarted","Data":"684437985c8f22998400743876ddb9bacb89eded4378fb5a9e57898883ec5076"}
Jul 30 19:56:14.186234 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:14.185844    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk"
Jul 30 19:56:14.203598 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:14.203542    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk" podStartSLOduration=1.717357524 podStartE2EDuration="2.203526045s" totalImagesPullingTime="486.168521ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:56:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:56:14.201535143 +0000 UTC m=+1902.166116224" watchObservedRunningTime="2026-07-30 19:56:14.203526045 +0000 UTC m=+1902.168107125"
Jul 30 19:56:20.198094 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:20.198060    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk"
Jul 30 19:56:26.871177 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:26.870288    2554 patch_prober.go:28] interesting pod/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": read tcp 10.132.0.2:52874->10.132.0.60:15021: read: connection reset by peer" start-of-body=
Jul 30 19:56:26.871177 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:26.870373    2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l" podUID="c514fd5e-706e-453f-859a-31c6d8609468" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": read tcp 10.132.0.2:52874->10.132.0.60:15021: read: connection reset by peer"
Jul 30 19:56:27.277021 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:27.276981    2554 generic.go:356] "Generic (PLEG): container finished" podID="c514fd5e-706e-453f-859a-31c6d8609468" containerID="41b243870a0f58263b3a518773801e367859b2c6eb1021873fba2ebac13e2bc1" exitCode=0
Jul 30 19:56:27.277197 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:27.277061    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l" event={"ID":"c514fd5e-706e-453f-859a-31c6d8609468","Type":"ContainerDied","Data":"41b243870a0f58263b3a518773801e367859b2c6eb1021873fba2ebac13e2bc1"}
Jul 30 19:56:27.277557 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:27.277542    2554 scope.go:117] "RemoveContainer" containerID="41b243870a0f58263b3a518773801e367859b2c6eb1021873fba2ebac13e2bc1"
Jul 30 19:56:27.278546 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:27.278519    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 19:56:27.278597 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:27.278584    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 19:56:27.278632 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:27.278617    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 19:56:28.285861 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:28.285826    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l" event={"ID":"c514fd5e-706e-453f-859a-31c6d8609468","Type":"ContainerStarted","Data":"1873879b6f47f20de9dae7352e087dc317f1bebdac7babf67267def955044546"}
Jul 30 19:56:29.229737 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:29.229697    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:29.229985 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:29.229833    2554 patch_prober.go:28] interesting pod/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 30 19:56:29.229985 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:29.229877    2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l" podUID="c514fd5e-706e-453f-859a-31c6d8609468" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 30 19:56:30.229477 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:30.229419    2554 patch_prober.go:28] interesting pod/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 30 19:56:30.230099 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:30.229535    2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l" podUID="c514fd5e-706e-453f-859a-31c6d8609468" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 30 19:56:31.229539 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:31.229500    2554 patch_prober.go:28] interesting pod/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 30 19:56:31.230706 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:31.230250    2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l" podUID="c514fd5e-706e-453f-859a-31c6d8609468" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 30 19:56:32.229702 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:32.229663    2554 patch_prober.go:28] interesting pod/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 30 19:56:32.230124 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:32.229738    2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l" podUID="c514fd5e-706e-453f-859a-31c6d8609468" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 30 19:56:33.232964 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:33.232917    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:33.233369 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:33.233204    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:33.233811 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:33.233795    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:37.119169 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:37.119136    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6d3e9f39-g9xr4"]
Jul 30 19:56:37.122715 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:37.122696    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6d3e9f39-g9xr4"
Jul 30 19:56:37.124940 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:37.124915    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7t7kk\""
Jul 30 19:56:37.131319 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:37.131015    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6d3e9f39-g9xr4"]
Jul 30 19:56:37.232393 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:37.232358    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdae4ecc-51ce-4732-8e24-4116bed6d75c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-6d3e9f39-g9xr4\" (UID: \"cdae4ecc-51ce-4732-8e24-4116bed6d75c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6d3e9f39-g9xr4"
Jul 30 19:56:37.333841 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:37.333806    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdae4ecc-51ce-4732-8e24-4116bed6d75c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-6d3e9f39-g9xr4\" (UID: \"cdae4ecc-51ce-4732-8e24-4116bed6d75c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6d3e9f39-g9xr4"
Jul 30 19:56:37.334547 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:37.334511    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdae4ecc-51ce-4732-8e24-4116bed6d75c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-6d3e9f39-g9xr4\" (UID: \"cdae4ecc-51ce-4732-8e24-4116bed6d75c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6d3e9f39-g9xr4"
Jul 30 19:56:37.436545 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:37.436450    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6d3e9f39-g9xr4"
Jul 30 19:56:37.580759 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:37.580733    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6d3e9f39-g9xr4"]
Jul 30 19:56:37.582165 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:56:37.582138    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdae4ecc_51ce_4732_8e24_4116bed6d75c.slice/crio-fbf8f84ca1b0c3426b17e9a70a6d1e205e4e074954f663f55e1607f67e9718b1 WatchSource:0}: Error finding container fbf8f84ca1b0c3426b17e9a70a6d1e205e4e074954f663f55e1607f67e9718b1: Status 404 returned error can't find the container with id fbf8f84ca1b0c3426b17e9a70a6d1e205e4e074954f663f55e1607f67e9718b1
Jul 30 19:56:38.344318 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:38.344286    2554 generic.go:356] "Generic (PLEG): container finished" podID="cdae4ecc-51ce-4732-8e24-4116bed6d75c" containerID="21db08122ec3ddf95940e122d9ccfb6366701c2505717e18801735b18995734d" exitCode=0
Jul 30 19:56:38.344644 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:38.344375    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6d3e9f39-g9xr4" event={"ID":"cdae4ecc-51ce-4732-8e24-4116bed6d75c","Type":"ContainerDied","Data":"21db08122ec3ddf95940e122d9ccfb6366701c2505717e18801735b18995734d"}
Jul 30 19:56:38.344644 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:38.344434    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6d3e9f39-g9xr4" event={"ID":"cdae4ecc-51ce-4732-8e24-4116bed6d75c","Type":"ContainerStarted","Data":"fbf8f84ca1b0c3426b17e9a70a6d1e205e4e074954f663f55e1607f67e9718b1"}
Jul 30 19:56:39.488357 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:39.488329    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6d3e9f39-g9xr4"
Jul 30 19:56:39.553730 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:39.553691    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdae4ecc-51ce-4732-8e24-4116bed6d75c-maas-api-service-ca\") pod \"cdae4ecc-51ce-4732-8e24-4116bed6d75c\" (UID: \"cdae4ecc-51ce-4732-8e24-4116bed6d75c\") "
Jul 30 19:56:39.554124 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:39.554101    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdae4ecc-51ce-4732-8e24-4116bed6d75c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cdae4ecc-51ce-4732-8e24-4116bed6d75c" (UID: "cdae4ecc-51ce-4732-8e24-4116bed6d75c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 19:56:39.655170 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:39.655079    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdae4ecc-51ce-4732-8e24-4116bed6d75c-maas-api-service-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:56:40.357947 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:40.357911    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6d3e9f39-g9xr4"
Jul 30 19:56:40.358160 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:40.357944    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6d3e9f39-g9xr4" event={"ID":"cdae4ecc-51ce-4732-8e24-4116bed6d75c","Type":"ContainerDied","Data":"fbf8f84ca1b0c3426b17e9a70a6d1e205e4e074954f663f55e1607f67e9718b1"}
Jul 30 19:56:40.358160 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:40.358012    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fbf8f84ca1b0c3426b17e9a70a6d1e205e4e074954f663f55e1607f67e9718b1"
Jul 30 19:56:43.254855 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:43.254821    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk"]
Jul 30 19:56:43.255330 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:43.255067    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk" podUID="828993bd-2909-479c-98a4-fe1961df9c4a" containerName="maas-api" containerID="cri-o://684437985c8f22998400743876ddb9bacb89eded4378fb5a9e57898883ec5076" gracePeriod=30
Jul 30 19:56:43.276776 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:43.276751    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:56:43.377585 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:43.377548    2554 generic.go:356] "Generic (PLEG): container finished" podID="828993bd-2909-479c-98a4-fe1961df9c4a" containerID="684437985c8f22998400743876ddb9bacb89eded4378fb5a9e57898883ec5076" exitCode=0
Jul 30 19:56:43.377733 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:43.377596    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk" event={"ID":"828993bd-2909-479c-98a4-fe1961df9c4a","Type":"ContainerDied","Data":"684437985c8f22998400743876ddb9bacb89eded4378fb5a9e57898883ec5076"}
Jul 30 19:56:43.437668 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:43.437602    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6d3e9f39-g9xr4"]
Jul 30 19:56:43.442330 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:43.442293    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6d3e9f39-g9xr4"]
Jul 30 19:56:43.526523 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:43.526498    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk"
Jul 30 19:56:43.570149 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:43.570106    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"]
Jul 30 19:56:43.570388 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:43.570361    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l" podUID="c514fd5e-706e-453f-859a-31c6d8609468" containerName="istio-proxy" containerID="cri-o://1873879b6f47f20de9dae7352e087dc317f1bebdac7babf67267def955044546" gracePeriod=30
Jul 30 19:56:43.593682 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:43.593654    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hw8v7\" (UniqueName: \"kubernetes.io/projected/828993bd-2909-479c-98a4-fe1961df9c4a-kube-api-access-hw8v7\") pod \"828993bd-2909-479c-98a4-fe1961df9c4a\" (UID: \"828993bd-2909-479c-98a4-fe1961df9c4a\") "
Jul 30 19:56:43.593851 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:43.593738    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/828993bd-2909-479c-98a4-fe1961df9c4a-openshift-service-ca\") pod \"828993bd-2909-479c-98a4-fe1961df9c4a\" (UID: \"828993bd-2909-479c-98a4-fe1961df9c4a\") "
Jul 30 19:56:43.593851 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:43.593808    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/828993bd-2909-479c-98a4-fe1961df9c4a-maas-api-tls\") pod \"828993bd-2909-479c-98a4-fe1961df9c4a\" (UID: \"828993bd-2909-479c-98a4-fe1961df9c4a\") "
Jul 30 19:56:43.594127 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:43.594104    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/828993bd-2909-479c-98a4-fe1961df9c4a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "828993bd-2909-479c-98a4-fe1961df9c4a" (UID: "828993bd-2909-479c-98a4-fe1961df9c4a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 19:56:43.595928 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:43.595900    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/828993bd-2909-479c-98a4-fe1961df9c4a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "828993bd-2909-479c-98a4-fe1961df9c4a" (UID: "828993bd-2909-479c-98a4-fe1961df9c4a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 19:56:43.596058 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:43.595969    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/828993bd-2909-479c-98a4-fe1961df9c4a-kube-api-access-hw8v7" (OuterVolumeSpecName: "kube-api-access-hw8v7") pod "828993bd-2909-479c-98a4-fe1961df9c4a" (UID: "828993bd-2909-479c-98a4-fe1961df9c4a"). InnerVolumeSpecName "kube-api-access-hw8v7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:56:43.695450 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:43.695412    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/828993bd-2909-479c-98a4-fe1961df9c4a-maas-api-tls\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:56:43.695450 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:43.695453    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hw8v7\" (UniqueName: \"kubernetes.io/projected/828993bd-2909-479c-98a4-fe1961df9c4a-kube-api-access-hw8v7\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:56:43.695661 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:43.695469    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/828993bd-2909-479c-98a4-fe1961df9c4a-openshift-service-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:56:44.384534 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:44.384500    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk"
Jul 30 19:56:44.385063 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:44.384533    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk" event={"ID":"828993bd-2909-479c-98a4-fe1961df9c4a","Type":"ContainerDied","Data":"956e0a1fef7ed953025421768efd855742a20766e7b9e0c35fc92a71d2a0142f"}
Jul 30 19:56:44.385063 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:44.384626    2554 scope.go:117] "RemoveContainer" containerID="684437985c8f22998400743876ddb9bacb89eded4378fb5a9e57898883ec5076"
Jul 30 19:56:44.406088 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:44.406058    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk"]
Jul 30 19:56:44.409682 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:44.409657    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6d3e9f39-845c9555c8-sh4nk"]
Jul 30 19:56:44.488578 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:44.488538    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="828993bd-2909-479c-98a4-fe1961df9c4a" path="/var/lib/kubelet/pods/828993bd-2909-479c-98a4-fe1961df9c4a/volumes"
Jul 30 19:56:44.488911 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:44.488895    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdae4ecc-51ce-4732-8e24-4116bed6d75c" path="/var/lib/kubelet/pods/cdae4ecc-51ce-4732-8e24-4116bed6d75c/volumes"
Jul 30 19:56:44.842003 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:44.841965    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:56:48.236314 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:48.236273    2554 patch_prober.go:28] interesting pod/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 30 19:56:48.236716 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:48.236351    2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l" podUID="c514fd5e-706e-453f-859a-31c6d8609468" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 19:56:48.834168 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:48.834092    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:48.946730 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:48.946688    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-istio-data\") pod \"c514fd5e-706e-453f-859a-31c6d8609468\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") "
Jul 30 19:56:48.946932 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:48.946748    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-credential-socket\") pod \"c514fd5e-706e-453f-859a-31c6d8609468\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") "
Jul 30 19:56:48.946932 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:48.946790    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-istio-envoy\") pod \"c514fd5e-706e-453f-859a-31c6d8609468\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") "
Jul 30 19:56:48.946932 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:48.946833    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c514fd5e-706e-453f-859a-31c6d8609468-istiod-ca-cert\") pod \"c514fd5e-706e-453f-859a-31c6d8609468\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") "
Jul 30 19:56:48.946932 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:48.946866    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74s5d\" (UniqueName: \"kubernetes.io/projected/c514fd5e-706e-453f-859a-31c6d8609468-kube-api-access-74s5d\") pod \"c514fd5e-706e-453f-859a-31c6d8609468\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") "
Jul 30 19:56:48.946932 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:48.946889    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c514fd5e-706e-453f-859a-31c6d8609468-istio-token\") pod \"c514fd5e-706e-453f-859a-31c6d8609468\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") "
Jul 30 19:56:48.947210 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:48.947050    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-workload-certs\") pod \"c514fd5e-706e-453f-859a-31c6d8609468\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") "
Jul 30 19:56:48.947210 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:48.947102    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c514fd5e-706e-453f-859a-31c6d8609468" (UID: "c514fd5e-706e-453f-859a-31c6d8609468"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:56:48.947210 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:48.947151    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c514fd5e-706e-453f-859a-31c6d8609468-istio-podinfo\") pod \"c514fd5e-706e-453f-859a-31c6d8609468\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") "
Jul 30 19:56:48.947210 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:48.947183    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-workload-socket\") pod \"c514fd5e-706e-453f-859a-31c6d8609468\" (UID: \"c514fd5e-706e-453f-859a-31c6d8609468\") "
Jul 30 19:56:48.947502 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:48.947428    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c514fd5e-706e-453f-859a-31c6d8609468" (UID: "c514fd5e-706e-453f-859a-31c6d8609468"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:56:48.947502 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:48.947465    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c514fd5e-706e-453f-859a-31c6d8609468-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c514fd5e-706e-453f-859a-31c6d8609468" (UID: "c514fd5e-706e-453f-859a-31c6d8609468"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 19:56:48.947725 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:48.947562    2554 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-credential-socket\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:56:48.947725 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:48.947612    2554 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c514fd5e-706e-453f-859a-31c6d8609468-istiod-ca-cert\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:56:48.947725 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:48.947630    2554 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-workload-certs\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:56:48.947725 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:48.947646    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c514fd5e-706e-453f-859a-31c6d8609468" (UID: "c514fd5e-706e-453f-859a-31c6d8609468"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:56:48.947725 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:48.947668    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-istio-data" (OuterVolumeSpecName: "istio-data") pod "c514fd5e-706e-453f-859a-31c6d8609468" (UID: "c514fd5e-706e-453f-859a-31c6d8609468"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:56:48.949382 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:48.949349    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c514fd5e-706e-453f-859a-31c6d8609468" (UID: "c514fd5e-706e-453f-859a-31c6d8609468"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:56:48.949481 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:48.949402    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c514fd5e-706e-453f-859a-31c6d8609468-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c514fd5e-706e-453f-859a-31c6d8609468" (UID: "c514fd5e-706e-453f-859a-31c6d8609468"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 19:56:48.949481 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:48.949443    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c514fd5e-706e-453f-859a-31c6d8609468-kube-api-access-74s5d" (OuterVolumeSpecName: "kube-api-access-74s5d") pod "c514fd5e-706e-453f-859a-31c6d8609468" (UID: "c514fd5e-706e-453f-859a-31c6d8609468"). InnerVolumeSpecName "kube-api-access-74s5d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:56:48.949481 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:48.949455    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c514fd5e-706e-453f-859a-31c6d8609468-istio-token" (OuterVolumeSpecName: "istio-token") pod "c514fd5e-706e-453f-859a-31c6d8609468" (UID: "c514fd5e-706e-453f-859a-31c6d8609468"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:56:49.048268 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.048225    2554 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-istio-envoy\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:56:49.048268 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.048261    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74s5d\" (UniqueName: \"kubernetes.io/projected/c514fd5e-706e-453f-859a-31c6d8609468-kube-api-access-74s5d\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:56:49.048268 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.048276    2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c514fd5e-706e-453f-859a-31c6d8609468-istio-token\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:56:49.048518 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.048288    2554 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c514fd5e-706e-453f-859a-31c6d8609468-istio-podinfo\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:56:49.048518 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.048299    2554 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-workload-socket\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:56:49.048518 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.048310    2554 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c514fd5e-706e-453f-859a-31c6d8609468-istio-data\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:56:49.310801 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.310757    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:56:49.379887 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.379854    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx"]
Jul 30 19:56:49.380303 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.380288    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="828993bd-2909-479c-98a4-fe1961df9c4a" containerName="maas-api"
Jul 30 19:56:49.380303 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.380304    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="828993bd-2909-479c-98a4-fe1961df9c4a" containerName="maas-api"
Jul 30 19:56:49.380396 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.380315    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c514fd5e-706e-453f-859a-31c6d8609468" containerName="istio-proxy"
Jul 30 19:56:49.380396 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.380322    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="c514fd5e-706e-453f-859a-31c6d8609468" containerName="istio-proxy"
Jul 30 19:56:49.380396 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.380342    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c514fd5e-706e-453f-859a-31c6d8609468" containerName="istio-proxy"
Jul 30 19:56:49.380396 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.380348    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="c514fd5e-706e-453f-859a-31c6d8609468" containerName="istio-proxy"
Jul 30 19:56:49.380396 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.380356    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdae4ecc-51ce-4732-8e24-4116bed6d75c" containerName="revoke-keys"
Jul 30 19:56:49.380396 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.380360    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdae4ecc-51ce-4732-8e24-4116bed6d75c" containerName="revoke-keys"
Jul 30 19:56:49.380579 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.380419    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="828993bd-2909-479c-98a4-fe1961df9c4a" containerName="maas-api"
Jul 30 19:56:49.380579 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.380427    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="c514fd5e-706e-453f-859a-31c6d8609468" containerName="istio-proxy"
Jul 30 19:56:49.380579 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.380434    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdae4ecc-51ce-4732-8e24-4116bed6d75c" containerName="revoke-keys"
Jul 30 19:56:49.380579 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.380441    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="c514fd5e-706e-453f-859a-31c6d8609468" containerName="istio-proxy"
Jul 30 19:56:49.383642 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.383625    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx"
Jul 30 19:56:49.386273 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.386251    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-57cec089-serving-cert\""
Jul 30 19:56:49.393138 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.393109    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx"]
Jul 30 19:56:49.410518 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.410479    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:56:49.417520 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.417486    2554 generic.go:356] "Generic (PLEG): container finished" podID="c514fd5e-706e-453f-859a-31c6d8609468" containerID="1873879b6f47f20de9dae7352e087dc317f1bebdac7babf67267def955044546" exitCode=0
Jul 30 19:56:49.417700 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.417570    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l" event={"ID":"c514fd5e-706e-453f-859a-31c6d8609468","Type":"ContainerDied","Data":"1873879b6f47f20de9dae7352e087dc317f1bebdac7babf67267def955044546"}
Jul 30 19:56:49.417700 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.417600    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"
Jul 30 19:56:49.417700 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.417651    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l" event={"ID":"c514fd5e-706e-453f-859a-31c6d8609468","Type":"ContainerDied","Data":"192c5d43e44757ff40d6477ca4eab72d5980f9e816cae6a2a70671e3dcecd968"}
Jul 30 19:56:49.417700 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.417688    2554 scope.go:117] "RemoveContainer" containerID="1873879b6f47f20de9dae7352e087dc317f1bebdac7babf67267def955044546"
Jul 30 19:56:49.429339 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.429314    2554 scope.go:117] "RemoveContainer" containerID="41b243870a0f58263b3a518773801e367859b2c6eb1021873fba2ebac13e2bc1"
Jul 30 19:56:49.439074 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.439050    2554 scope.go:117] "RemoveContainer" containerID="1873879b6f47f20de9dae7352e087dc317f1bebdac7babf67267def955044546"
Jul 30 19:56:49.439432 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:56:49.439399    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1873879b6f47f20de9dae7352e087dc317f1bebdac7babf67267def955044546\": container with ID starting with 1873879b6f47f20de9dae7352e087dc317f1bebdac7babf67267def955044546 not found: ID does not exist" containerID="1873879b6f47f20de9dae7352e087dc317f1bebdac7babf67267def955044546"
Jul 30 19:56:49.439538 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.439444    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1873879b6f47f20de9dae7352e087dc317f1bebdac7babf67267def955044546"} err="failed to get container status \"1873879b6f47f20de9dae7352e087dc317f1bebdac7babf67267def955044546\": rpc error: code = NotFound desc = could not find container \"1873879b6f47f20de9dae7352e087dc317f1bebdac7babf67267def955044546\": container with ID starting with 1873879b6f47f20de9dae7352e087dc317f1bebdac7babf67267def955044546 not found: ID does not exist"
Jul 30 19:56:49.439538 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.439468    2554 scope.go:117] "RemoveContainer" containerID="41b243870a0f58263b3a518773801e367859b2c6eb1021873fba2ebac13e2bc1"
Jul 30 19:56:49.440004 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:56:49.439925    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41b243870a0f58263b3a518773801e367859b2c6eb1021873fba2ebac13e2bc1\": container with ID starting with 41b243870a0f58263b3a518773801e367859b2c6eb1021873fba2ebac13e2bc1 not found: ID does not exist" containerID="41b243870a0f58263b3a518773801e367859b2c6eb1021873fba2ebac13e2bc1"
Jul 30 19:56:49.440004 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.440015    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41b243870a0f58263b3a518773801e367859b2c6eb1021873fba2ebac13e2bc1"} err="failed to get container status \"41b243870a0f58263b3a518773801e367859b2c6eb1021873fba2ebac13e2bc1\": rpc error: code = NotFound desc = could not find container \"41b243870a0f58263b3a518773801e367859b2c6eb1021873fba2ebac13e2bc1\": container with ID starting with 41b243870a0f58263b3a518773801e367859b2c6eb1021873fba2ebac13e2bc1 not found: ID does not exist"
Jul 30 19:56:49.443042 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.443017    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"]
Jul 30 19:56:49.447078 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.447055    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-6d3e9f39-openshift-default-554b758dbb-gmm4l"]
Jul 30 19:56:49.514622 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.514570    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:56:49.553222 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.553183    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5899946f-9064-4cd5-a457-014dfe19d3e6-maas-api-tls\") pod \"maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx\" (UID: \"5899946f-9064-4cd5-a457-014dfe19d3e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx"
Jul 30 19:56:49.553403 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.553238    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5899946f-9064-4cd5-a457-014dfe19d3e6-openshift-service-ca\") pod \"maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx\" (UID: \"5899946f-9064-4cd5-a457-014dfe19d3e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx"
Jul 30 19:56:49.553403 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.553332    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z5b2\" (UniqueName: \"kubernetes.io/projected/5899946f-9064-4cd5-a457-014dfe19d3e6-kube-api-access-8z5b2\") pod \"maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx\" (UID: \"5899946f-9064-4cd5-a457-014dfe19d3e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx"
Jul 30 19:56:49.654602 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.654491    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5899946f-9064-4cd5-a457-014dfe19d3e6-maas-api-tls\") pod \"maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx\" (UID: \"5899946f-9064-4cd5-a457-014dfe19d3e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx"
Jul 30 19:56:49.654602 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.654556    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5899946f-9064-4cd5-a457-014dfe19d3e6-openshift-service-ca\") pod \"maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx\" (UID: \"5899946f-9064-4cd5-a457-014dfe19d3e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx"
Jul 30 19:56:49.654843 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.654655    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z5b2\" (UniqueName: \"kubernetes.io/projected/5899946f-9064-4cd5-a457-014dfe19d3e6-kube-api-access-8z5b2\") pod \"maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx\" (UID: \"5899946f-9064-4cd5-a457-014dfe19d3e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx"
Jul 30 19:56:49.655311 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.655292    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5899946f-9064-4cd5-a457-014dfe19d3e6-openshift-service-ca\") pod \"maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx\" (UID: \"5899946f-9064-4cd5-a457-014dfe19d3e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx"
Jul 30 19:56:49.657023 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.657001    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5899946f-9064-4cd5-a457-014dfe19d3e6-maas-api-tls\") pod \"maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx\" (UID: \"5899946f-9064-4cd5-a457-014dfe19d3e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx"
Jul 30 19:56:49.670021 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.669991    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z5b2\" (UniqueName: \"kubernetes.io/projected/5899946f-9064-4cd5-a457-014dfe19d3e6-kube-api-access-8z5b2\") pod \"maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx\" (UID: \"5899946f-9064-4cd5-a457-014dfe19d3e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx"
Jul 30 19:56:49.696937 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.696896    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx"
Jul 30 19:56:49.850622 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.850587    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx"]
Jul 30 19:56:49.852264 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:56:49.852238    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5899946f_9064_4cd5_a457_014dfe19d3e6.slice/crio-3a90b9c54b007b41de669708c18da413686c51cb54e1e4e0a9c996e4a04d4cee WatchSource:0}: Error finding container 3a90b9c54b007b41de669708c18da413686c51cb54e1e4e0a9c996e4a04d4cee: Status 404 returned error can't find the container with id 3a90b9c54b007b41de669708c18da413686c51cb54e1e4e0a9c996e4a04d4cee
Jul 30 19:56:49.955273 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:49.955184    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:56:50.060993 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:50.060948    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:56:50.427012 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:50.426972    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx" event={"ID":"5899946f-9064-4cd5-a457-014dfe19d3e6","Type":"ContainerStarted","Data":"98b181f90000e97bfafd46cdb4e924db6259045e30c70f254106e7d46439bf4a"}
Jul 30 19:56:50.427490 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:50.427037    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx" event={"ID":"5899946f-9064-4cd5-a457-014dfe19d3e6","Type":"ContainerStarted","Data":"3a90b9c54b007b41de669708c18da413686c51cb54e1e4e0a9c996e4a04d4cee"}
Jul 30 19:56:50.427490 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:50.427069    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx"
Jul 30 19:56:50.445768 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:50.445704    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx" podStartSLOduration=1.025308271 podStartE2EDuration="1.445684934s" totalImagesPullingTime="420.376663ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:56:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:56:50.443050325 +0000 UTC m=+1938.407631419" watchObservedRunningTime="2026-07-30 19:56:50.445684934 +0000 UTC m=+1938.410266015"
Jul 30 19:56:50.488288 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:50.488251    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c514fd5e-706e-453f-859a-31c6d8609468" path="/var/lib/kubelet/pods/c514fd5e-706e-453f-859a-31c6d8609468/volumes"
Jul 30 19:56:56.440013 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:56:56.439980    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx"
Jul 30 19:57:14.325094 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:14.325059    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-57cec089-crh2z"]
Jul 30 19:57:14.328742 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:14.328723    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-57cec089-crh2z"
Jul 30 19:57:14.331587 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:14.331559    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7t7kk\""
Jul 30 19:57:14.336877 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:14.336853    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-57cec089-crh2z"]
Jul 30 19:57:14.391512 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:14.391474    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c520ac4-b4c5-41a1-9818-b89736a6ca5e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-57cec089-crh2z\" (UID: \"0c520ac4-b4c5-41a1-9818-b89736a6ca5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-57cec089-crh2z"
Jul 30 19:57:14.492371 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:14.492335    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c520ac4-b4c5-41a1-9818-b89736a6ca5e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-57cec089-crh2z\" (UID: \"0c520ac4-b4c5-41a1-9818-b89736a6ca5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-57cec089-crh2z"
Jul 30 19:57:14.493014 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:14.492996    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c520ac4-b4c5-41a1-9818-b89736a6ca5e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-57cec089-crh2z\" (UID: \"0c520ac4-b4c5-41a1-9818-b89736a6ca5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-57cec089-crh2z"
Jul 30 19:57:14.642700 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:14.642618    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-57cec089-crh2z"
Jul 30 19:57:14.782343 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:14.782314    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-57cec089-crh2z"]
Jul 30 19:57:14.785862 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:57:14.785834    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c520ac4_b4c5_41a1_9818_b89736a6ca5e.slice/crio-a4d103a2c534c579c7d63c4a6d29800fdbcb3bc92d3b036c7a301f0edf547d96 WatchSource:0}: Error finding container a4d103a2c534c579c7d63c4a6d29800fdbcb3bc92d3b036c7a301f0edf547d96: Status 404 returned error can't find the container with id a4d103a2c534c579c7d63c4a6d29800fdbcb3bc92d3b036c7a301f0edf547d96
Jul 30 19:57:15.581723 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:15.581680    2554 generic.go:356] "Generic (PLEG): container finished" podID="0c520ac4-b4c5-41a1-9818-b89736a6ca5e" containerID="6bfc887a343607fedb1d9cdf88816e0f6b20459c8dbe7c2c5c969cdd9ab899af" exitCode=0
Jul 30 19:57:15.582162 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:15.581722    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-57cec089-crh2z" event={"ID":"0c520ac4-b4c5-41a1-9818-b89736a6ca5e","Type":"ContainerDied","Data":"6bfc887a343607fedb1d9cdf88816e0f6b20459c8dbe7c2c5c969cdd9ab899af"}
Jul 30 19:57:15.582162 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:15.581777    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-57cec089-crh2z" event={"ID":"0c520ac4-b4c5-41a1-9818-b89736a6ca5e","Type":"ContainerStarted","Data":"a4d103a2c534c579c7d63c4a6d29800fdbcb3bc92d3b036c7a301f0edf547d96"}
Jul 30 19:57:16.722534 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:16.722509    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-57cec089-crh2z"
Jul 30 19:57:16.818983 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:16.818938    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c520ac4-b4c5-41a1-9818-b89736a6ca5e-maas-api-service-ca\") pod \"0c520ac4-b4c5-41a1-9818-b89736a6ca5e\" (UID: \"0c520ac4-b4c5-41a1-9818-b89736a6ca5e\") "
Jul 30 19:57:16.819258 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:16.819237    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c520ac4-b4c5-41a1-9818-b89736a6ca5e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0c520ac4-b4c5-41a1-9818-b89736a6ca5e" (UID: "0c520ac4-b4c5-41a1-9818-b89736a6ca5e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 19:57:16.920178 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:16.920086    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c520ac4-b4c5-41a1-9818-b89736a6ca5e-maas-api-service-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:57:17.594163 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:17.594132    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-57cec089-crh2z"
Jul 30 19:57:17.594350 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:17.594167    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-57cec089-crh2z" event={"ID":"0c520ac4-b4c5-41a1-9818-b89736a6ca5e","Type":"ContainerDied","Data":"a4d103a2c534c579c7d63c4a6d29800fdbcb3bc92d3b036c7a301f0edf547d96"}
Jul 30 19:57:17.594350 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:17.594217    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a4d103a2c534c579c7d63c4a6d29800fdbcb3bc92d3b036c7a301f0edf547d96"
Jul 30 19:57:20.840755 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:20.840715    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx"]
Jul 30 19:57:20.841191 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:20.841007    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx" podUID="5899946f-9064-4cd5-a457-014dfe19d3e6" containerName="maas-api" containerID="cri-o://98b181f90000e97bfafd46cdb4e924db6259045e30c70f254106e7d46439bf4a" gracePeriod=30
Jul 30 19:57:20.854716 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:20.854685    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:57:20.927553 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:20.927510    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:57:20.978180 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:20.978145    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:57:21.024177 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.024109    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-57cec089-crh2z"]
Jul 30 19:57:21.027396 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.027363    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-57cec089-crh2z"]
Jul 30 19:57:21.028726 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.028688    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:57:21.104607 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.104548    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx"
Jul 30 19:57:21.161092 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.161064    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5899946f-9064-4cd5-a457-014dfe19d3e6-maas-api-tls\") pod \"5899946f-9064-4cd5-a457-014dfe19d3e6\" (UID: \"5899946f-9064-4cd5-a457-014dfe19d3e6\") "
Jul 30 19:57:21.161285 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.161110    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5899946f-9064-4cd5-a457-014dfe19d3e6-openshift-service-ca\") pod \"5899946f-9064-4cd5-a457-014dfe19d3e6\" (UID: \"5899946f-9064-4cd5-a457-014dfe19d3e6\") "
Jul 30 19:57:21.161285 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.161215    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8z5b2\" (UniqueName: \"kubernetes.io/projected/5899946f-9064-4cd5-a457-014dfe19d3e6-kube-api-access-8z5b2\") pod \"5899946f-9064-4cd5-a457-014dfe19d3e6\" (UID: \"5899946f-9064-4cd5-a457-014dfe19d3e6\") "
Jul 30 19:57:21.161574 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.161548    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5899946f-9064-4cd5-a457-014dfe19d3e6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5899946f-9064-4cd5-a457-014dfe19d3e6" (UID: "5899946f-9064-4cd5-a457-014dfe19d3e6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 19:57:21.165122 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.165093    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5899946f-9064-4cd5-a457-014dfe19d3e6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5899946f-9064-4cd5-a457-014dfe19d3e6" (UID: "5899946f-9064-4cd5-a457-014dfe19d3e6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 19:57:21.165253 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.165215    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5899946f-9064-4cd5-a457-014dfe19d3e6-kube-api-access-8z5b2" (OuterVolumeSpecName: "kube-api-access-8z5b2") pod "5899946f-9064-4cd5-a457-014dfe19d3e6" (UID: "5899946f-9064-4cd5-a457-014dfe19d3e6"). InnerVolumeSpecName "kube-api-access-8z5b2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:57:21.166333 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.166309    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:57:21.262619 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.262584    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8z5b2\" (UniqueName: \"kubernetes.io/projected/5899946f-9064-4cd5-a457-014dfe19d3e6-kube-api-access-8z5b2\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:57:21.262619 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.262617    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5899946f-9064-4cd5-a457-014dfe19d3e6-maas-api-tls\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:57:21.262824 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.262631    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5899946f-9064-4cd5-a457-014dfe19d3e6-openshift-service-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:57:21.423114 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.423019    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:57:21.619619 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.619582    2554 generic.go:356] "Generic (PLEG): container finished" podID="5899946f-9064-4cd5-a457-014dfe19d3e6" containerID="98b181f90000e97bfafd46cdb4e924db6259045e30c70f254106e7d46439bf4a" exitCode=0
Jul 30 19:57:21.619619 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.619619    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx"
Jul 30 19:57:21.619880 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.619664    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx" event={"ID":"5899946f-9064-4cd5-a457-014dfe19d3e6","Type":"ContainerDied","Data":"98b181f90000e97bfafd46cdb4e924db6259045e30c70f254106e7d46439bf4a"}
Jul 30 19:57:21.619880 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.619727    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx" event={"ID":"5899946f-9064-4cd5-a457-014dfe19d3e6","Type":"ContainerDied","Data":"3a90b9c54b007b41de669708c18da413686c51cb54e1e4e0a9c996e4a04d4cee"}
Jul 30 19:57:21.619880 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.619751    2554 scope.go:117] "RemoveContainer" containerID="98b181f90000e97bfafd46cdb4e924db6259045e30c70f254106e7d46439bf4a"
Jul 30 19:57:21.629710 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.629689    2554 scope.go:117] "RemoveContainer" containerID="98b181f90000e97bfafd46cdb4e924db6259045e30c70f254106e7d46439bf4a"
Jul 30 19:57:21.629991 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:57:21.629970    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98b181f90000e97bfafd46cdb4e924db6259045e30c70f254106e7d46439bf4a\": container with ID starting with 98b181f90000e97bfafd46cdb4e924db6259045e30c70f254106e7d46439bf4a not found: ID does not exist" containerID="98b181f90000e97bfafd46cdb4e924db6259045e30c70f254106e7d46439bf4a"
Jul 30 19:57:21.630071 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.630001    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98b181f90000e97bfafd46cdb4e924db6259045e30c70f254106e7d46439bf4a"} err="failed to get container status \"98b181f90000e97bfafd46cdb4e924db6259045e30c70f254106e7d46439bf4a\": rpc error: code = NotFound desc = could not find container \"98b181f90000e97bfafd46cdb4e924db6259045e30c70f254106e7d46439bf4a\": container with ID starting with 98b181f90000e97bfafd46cdb4e924db6259045e30c70f254106e7d46439bf4a not found: ID does not exist"
Jul 30 19:57:21.639807 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.639778    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:57:21.641516 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.641494    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx"]
Jul 30 19:57:21.644279 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.644256    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-57cec089-547cdc8bd6-v2rmx"]
Jul 30 19:57:21.926098 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.926066    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"]
Jul 30 19:57:21.926471 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.926462    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5899946f-9064-4cd5-a457-014dfe19d3e6" containerName="maas-api"
Jul 30 19:57:21.926517 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.926473    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="5899946f-9064-4cd5-a457-014dfe19d3e6" containerName="maas-api"
Jul 30 19:57:21.926517 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.926489    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c520ac4-b4c5-41a1-9818-b89736a6ca5e" containerName="revoke-keys"
Jul 30 19:57:21.926517 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.926494    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c520ac4-b4c5-41a1-9818-b89736a6ca5e" containerName="revoke-keys"
Jul 30 19:57:21.926613 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.926556    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="5899946f-9064-4cd5-a457-014dfe19d3e6" containerName="maas-api"
Jul 30 19:57:21.926613 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.926565    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c520ac4-b4c5-41a1-9818-b89736a6ca5e" containerName="revoke-keys"
Jul 30 19:57:21.931062 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.931041    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:21.933365 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.933344    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a76d0719-openshift-default-dockercfg-b5fql\""
Jul 30 19:57:21.942930 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.942718    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"]
Jul 30 19:57:21.957088 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:21.957057    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:57:22.012138 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.012103    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:57:22.069734 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.069700    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v9pc\" (UniqueName: \"kubernetes.io/projected/9742486a-2ba8-487a-b00d-2e415649ff90-kube-api-access-4v9pc\") pod \"e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") " pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:22.069889 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.069753    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9742486a-2ba8-487a-b00d-2e415649ff90-istio-podinfo\") pod \"e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") " pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:22.069889 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.069844    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-istio-envoy\") pod \"e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") " pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:22.069889 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.069883    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-credential-socket\") pod \"e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") " pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:22.070027 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.069919    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9742486a-2ba8-487a-b00d-2e415649ff90-istio-token\") pod \"e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") " pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:22.070068 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.070027    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-workload-certs\") pod \"e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") " pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:22.070103 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.070070    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9742486a-2ba8-487a-b00d-2e415649ff90-istiod-ca-cert\") pod \"e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") " pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:22.070103 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.070090    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-istio-data\") pod \"e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") " pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:22.070168 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.070150    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-workload-socket\") pod \"e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") " pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:22.170857 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.170826    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-workload-certs\") pod \"e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") " pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:22.170857 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.170866    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9742486a-2ba8-487a-b00d-2e415649ff90-istiod-ca-cert\") pod \"e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") " pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:22.171082 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.170889    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-istio-data\") pod \"e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") " pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:22.171082 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.170914    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-workload-socket\") pod \"e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") " pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:22.171082 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.170990    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v9pc\" (UniqueName: \"kubernetes.io/projected/9742486a-2ba8-487a-b00d-2e415649ff90-kube-api-access-4v9pc\") pod \"e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") " pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:22.171082 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.171023    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9742486a-2ba8-487a-b00d-2e415649ff90-istio-podinfo\") pod \"e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") " pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:22.171082 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.171069    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-istio-envoy\") pod \"e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") " pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:22.171332 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.171097    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-credential-socket\") pod \"e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") " pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:22.171332 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.171125    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9742486a-2ba8-487a-b00d-2e415649ff90-istio-token\") pod \"e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") " pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:22.171332 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.171293    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-workload-certs\") pod \"e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") " pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:22.171475 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.171380    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-istio-data\") pod \"e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") " pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:22.171475 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.171397    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-workload-socket\") pod \"e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") " pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:22.171570 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.171552    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-credential-socket\") pod \"e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") " pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:22.171877 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.171849    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9742486a-2ba8-487a-b00d-2e415649ff90-istiod-ca-cert\") pod \"e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") " pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:22.173532 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.173502    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-istio-envoy\") pod \"e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") " pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:22.173740 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.173721    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9742486a-2ba8-487a-b00d-2e415649ff90-istio-podinfo\") pod \"e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") " pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:22.179112 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.179056    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9742486a-2ba8-487a-b00d-2e415649ff90-istio-token\") pod \"e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") " pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:22.179297 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.179272    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v9pc\" (UniqueName: \"kubernetes.io/projected/9742486a-2ba8-487a-b00d-2e415649ff90-kube-api-access-4v9pc\") pod \"e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") " pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:22.243436 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.243396    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:22.384183 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.384150    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"]
Jul 30 19:57:22.385486 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:57:22.385459    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9742486a_2ba8_487a_b00d_2e415649ff90.slice/crio-329ec1823a3a51d1480dcd7956c6e399d331cc833f9533fc6a91363a2bb4234b WatchSource:0}: Error finding container 329ec1823a3a51d1480dcd7956c6e399d331cc833f9533fc6a91363a2bb4234b: Status 404 returned error can't find the container with id 329ec1823a3a51d1480dcd7956c6e399d331cc833f9533fc6a91363a2bb4234b
Jul 30 19:57:22.387636 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.387605    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 19:57:22.387719 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.387688    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 19:57:22.387761 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.387721    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 19:57:22.488811 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.488771    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c520ac4-b4c5-41a1-9818-b89736a6ca5e" path="/var/lib/kubelet/pods/0c520ac4-b4c5-41a1-9818-b89736a6ca5e/volumes"
Jul 30 19:57:22.489262 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.489240    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5899946f-9064-4cd5-a457-014dfe19d3e6" path="/var/lib/kubelet/pods/5899946f-9064-4cd5-a457-014dfe19d3e6/volumes"
Jul 30 19:57:22.638322 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.638278    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k" event={"ID":"9742486a-2ba8-487a-b00d-2e415649ff90","Type":"ContainerStarted","Data":"84eeec88782c8eef8c77dca30b4b1be514738d5a90b7a46769455a03396dcd6f"}
Jul 30 19:57:22.638469 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.638380    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k" event={"ID":"9742486a-2ba8-487a-b00d-2e415649ff90","Type":"ContainerStarted","Data":"329ec1823a3a51d1480dcd7956c6e399d331cc833f9533fc6a91363a2bb4234b"}
Jul 30 19:57:22.657502 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:22.657449    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k" podStartSLOduration=1.657429796 podStartE2EDuration="1.657429796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:57:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:57:22.65615983 +0000 UTC m=+1970.620740914" watchObservedRunningTime="2026-07-30 19:57:22.657429796 +0000 UTC m=+1970.622010878"
Jul 30 19:57:23.243875 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:23.243836    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:23.248854 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:23.248829    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:23.644319 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:23.644227    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:23.645180 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:23.645158    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:24.044336 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:24.044297    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:57:24.144694 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:24.144655    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:57:25.252768 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:25.252731    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc"]
Jul 30 19:57:25.257589 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:25.257566    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc"
Jul 30 19:57:25.259862 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:25.259841    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-a76d0719-serving-cert\""
Jul 30 19:57:25.267900 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:25.267873    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc"]
Jul 30 19:57:25.279976 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:25.279931    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:57:25.387905 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:25.387868    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:57:25.401224 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:25.401195    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zjrx\" (UniqueName: \"kubernetes.io/projected/3215b1b5-1f6b-4003-b2b1-c786acff3284-kube-api-access-9zjrx\") pod \"maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc\" (UID: \"3215b1b5-1f6b-4003-b2b1-c786acff3284\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc"
Jul 30 19:57:25.401314 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:25.401279    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3215b1b5-1f6b-4003-b2b1-c786acff3284-openshift-service-ca\") pod \"maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc\" (UID: \"3215b1b5-1f6b-4003-b2b1-c786acff3284\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc"
Jul 30 19:57:25.401314 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:25.401300    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3215b1b5-1f6b-4003-b2b1-c786acff3284-maas-api-tls\") pod \"maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc\" (UID: \"3215b1b5-1f6b-4003-b2b1-c786acff3284\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc"
Jul 30 19:57:25.502671 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:25.502628    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zjrx\" (UniqueName: \"kubernetes.io/projected/3215b1b5-1f6b-4003-b2b1-c786acff3284-kube-api-access-9zjrx\") pod \"maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc\" (UID: \"3215b1b5-1f6b-4003-b2b1-c786acff3284\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc"
Jul 30 19:57:25.502876 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:25.502692    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3215b1b5-1f6b-4003-b2b1-c786acff3284-openshift-service-ca\") pod \"maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc\" (UID: \"3215b1b5-1f6b-4003-b2b1-c786acff3284\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc"
Jul 30 19:57:25.502876 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:25.502713    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3215b1b5-1f6b-4003-b2b1-c786acff3284-maas-api-tls\") pod \"maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc\" (UID: \"3215b1b5-1f6b-4003-b2b1-c786acff3284\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc"
Jul 30 19:57:25.503345 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:25.503322    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3215b1b5-1f6b-4003-b2b1-c786acff3284-openshift-service-ca\") pod \"maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc\" (UID: \"3215b1b5-1f6b-4003-b2b1-c786acff3284\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc"
Jul 30 19:57:25.505116 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:25.505098    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3215b1b5-1f6b-4003-b2b1-c786acff3284-maas-api-tls\") pod \"maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc\" (UID: \"3215b1b5-1f6b-4003-b2b1-c786acff3284\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc"
Jul 30 19:57:25.511257 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:25.511229    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zjrx\" (UniqueName: \"kubernetes.io/projected/3215b1b5-1f6b-4003-b2b1-c786acff3284-kube-api-access-9zjrx\") pod \"maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc\" (UID: \"3215b1b5-1f6b-4003-b2b1-c786acff3284\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc"
Jul 30 19:57:25.570462 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:25.570425    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc"
Jul 30 19:57:25.723583 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:25.723481    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc"]
Jul 30 19:57:25.723918 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:57:25.723891    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3215b1b5_1f6b_4003_b2b1_c786acff3284.slice/crio-53699223a053765e8bc9df50ad7b374726df0bf9b53f6f5724aaf0af165f0c74 WatchSource:0}: Error finding container 53699223a053765e8bc9df50ad7b374726df0bf9b53f6f5724aaf0af165f0c74: Status 404 returned error can't find the container with id 53699223a053765e8bc9df50ad7b374726df0bf9b53f6f5724aaf0af165f0c74
Jul 30 19:57:25.756632 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:25.756533    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:57:25.831129 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:25.831101    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:57:26.664506 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:26.664472    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc" event={"ID":"3215b1b5-1f6b-4003-b2b1-c786acff3284","Type":"ContainerStarted","Data":"ded9c6f2fc533425b6f4c6ab316cc61c534a4a57a0877bbf88e458ed389440bc"}
Jul 30 19:57:26.664938 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:26.664523    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc" event={"ID":"3215b1b5-1f6b-4003-b2b1-c786acff3284","Type":"ContainerStarted","Data":"53699223a053765e8bc9df50ad7b374726df0bf9b53f6f5724aaf0af165f0c74"}
Jul 30 19:57:26.664938 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:26.664620    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc"
Jul 30 19:57:26.681870 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:26.681814    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc" podStartSLOduration=1.24268522 podStartE2EDuration="1.681797152s" totalImagesPullingTime="439.111932ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:57:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:57:26.680376579 +0000 UTC m=+1974.644957661" watchObservedRunningTime="2026-07-30 19:57:26.681797152 +0000 UTC m=+1974.646378233"
Jul 30 19:57:32.676211 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:32.676178    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc"
Jul 30 19:57:38.874544 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:38.874499    2554 patch_prober.go:28] interesting pod/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": read tcp 10.132.0.2:43700->10.132.0.65:15021: read: connection reset by peer" start-of-body=
Jul 30 19:57:38.874990 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:38.874564    2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k" podUID="9742486a-2ba8-487a-b00d-2e415649ff90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": read tcp 10.132.0.2:43700->10.132.0.65:15021: read: connection reset by peer"
Jul 30 19:57:39.757020 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:39.756984    2554 generic.go:356] "Generic (PLEG): container finished" podID="9742486a-2ba8-487a-b00d-2e415649ff90" containerID="84eeec88782c8eef8c77dca30b4b1be514738d5a90b7a46769455a03396dcd6f" exitCode=0
Jul 30 19:57:39.757195 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:39.757058    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k" event={"ID":"9742486a-2ba8-487a-b00d-2e415649ff90","Type":"ContainerDied","Data":"84eeec88782c8eef8c77dca30b4b1be514738d5a90b7a46769455a03396dcd6f"}
Jul 30 19:57:39.757463 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:39.757450    2554 scope.go:117] "RemoveContainer" containerID="84eeec88782c8eef8c77dca30b4b1be514738d5a90b7a46769455a03396dcd6f"
Jul 30 19:57:39.758436 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:39.758406    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 19:57:39.758488 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:39.758476    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 19:57:39.758529 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:39.758512    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 19:57:40.764791 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:40.764755    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k" event={"ID":"9742486a-2ba8-487a-b00d-2e415649ff90","Type":"ContainerStarted","Data":"2f73999e8c7e79c7dc7c9de8e3e2a1dc4f8f010d5c1070db4321074940f06dff"}
Jul 30 19:57:41.243854 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:41.243812    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:41.244094 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:41.244073    2554 patch_prober.go:28] interesting pod/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Jul 30 19:57:41.244145 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:41.244119    2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k" podUID="9742486a-2ba8-487a-b00d-2e415649ff90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Jul 30 19:57:42.244796 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:42.244757    2554 patch_prober.go:28] interesting pod/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Jul 30 19:57:42.245204 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:42.244817    2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k" podUID="9742486a-2ba8-487a-b00d-2e415649ff90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Jul 30 19:57:43.244246 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:43.244198    2554 patch_prober.go:28] interesting pod/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Jul 30 19:57:43.244460 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:43.244285    2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k" podUID="9742486a-2ba8-487a-b00d-2e415649ff90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Jul 30 19:57:44.243869 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:44.243831    2554 patch_prober.go:28] interesting pod/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Jul 30 19:57:44.244314 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:44.243889    2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k" podUID="9742486a-2ba8-487a-b00d-2e415649ff90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Jul 30 19:57:45.247208 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:45.247175    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:45.247768 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:45.247642    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:45.248390 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:45.248369    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:57:49.813969 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:49.813909    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a76d0719-nsdz4"]
Jul 30 19:57:49.816701 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:49.816680    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a76d0719-nsdz4"
Jul 30 19:57:49.819020 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:49.818997    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7t7kk\""
Jul 30 19:57:49.825584 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:49.825556    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a76d0719-nsdz4"]
Jul 30 19:57:49.832160 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:49.832126    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e68a861-d676-4ebc-8b1b-3adf489b7296-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a76d0719-nsdz4\" (UID: \"5e68a861-d676-4ebc-8b1b-3adf489b7296\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a76d0719-nsdz4"
Jul 30 19:57:49.932841 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:49.932805    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e68a861-d676-4ebc-8b1b-3adf489b7296-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a76d0719-nsdz4\" (UID: \"5e68a861-d676-4ebc-8b1b-3adf489b7296\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a76d0719-nsdz4"
Jul 30 19:57:49.933478 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:49.933460    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e68a861-d676-4ebc-8b1b-3adf489b7296-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a76d0719-nsdz4\" (UID: \"5e68a861-d676-4ebc-8b1b-3adf489b7296\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a76d0719-nsdz4"
Jul 30 19:57:50.129444 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:50.129361    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a76d0719-nsdz4"
Jul 30 19:57:50.267358 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:50.267331    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a76d0719-nsdz4"]
Jul 30 19:57:50.269565 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:57:50.269523    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e68a861_d676_4ebc_8b1b_3adf489b7296.slice/crio-71209920a994fe9c51192000f89ab5a193c9041c05440f5c8dc41170c7e85e01 WatchSource:0}: Error finding container 71209920a994fe9c51192000f89ab5a193c9041c05440f5c8dc41170c7e85e01: Status 404 returned error can't find the container with id 71209920a994fe9c51192000f89ab5a193c9041c05440f5c8dc41170c7e85e01
Jul 30 19:57:50.828741 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:50.828703    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a76d0719-nsdz4" event={"ID":"5e68a861-d676-4ebc-8b1b-3adf489b7296","Type":"ContainerStarted","Data":"e60ae343ce5cbc8deeda46d6be08e329d75931ec9cee1f727a5ffb70a053d681"}
Jul 30 19:57:50.829203 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:50.828775    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a76d0719-nsdz4" event={"ID":"5e68a861-d676-4ebc-8b1b-3adf489b7296","Type":"ContainerStarted","Data":"71209920a994fe9c51192000f89ab5a193c9041c05440f5c8dc41170c7e85e01"}
Jul 30 19:57:50.844456 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:50.844413    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a76d0719-nsdz4" podStartSLOduration=1.8443973869999999 podStartE2EDuration="1.844397387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:57:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:57:50.842779762 +0000 UTC m=+1998.807360843" watchObservedRunningTime="2026-07-30 19:57:50.844397387 +0000 UTC m=+1998.808978485"
Jul 30 19:57:51.836134 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:51.836096    2554 generic.go:356] "Generic (PLEG): container finished" podID="5e68a861-d676-4ebc-8b1b-3adf489b7296" containerID="e60ae343ce5cbc8deeda46d6be08e329d75931ec9cee1f727a5ffb70a053d681" exitCode=0
Jul 30 19:57:51.836598 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:51.836146    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a76d0719-nsdz4" event={"ID":"5e68a861-d676-4ebc-8b1b-3adf489b7296","Type":"ContainerDied","Data":"e60ae343ce5cbc8deeda46d6be08e329d75931ec9cee1f727a5ffb70a053d681"}
Jul 30 19:57:52.976629 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:52.976598    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a76d0719-nsdz4"
Jul 30 19:57:53.064279 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:53.064247    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e68a861-d676-4ebc-8b1b-3adf489b7296-maas-api-service-ca\") pod \"5e68a861-d676-4ebc-8b1b-3adf489b7296\" (UID: \"5e68a861-d676-4ebc-8b1b-3adf489b7296\") "
Jul 30 19:57:53.064586 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:53.064566    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e68a861-d676-4ebc-8b1b-3adf489b7296-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5e68a861-d676-4ebc-8b1b-3adf489b7296" (UID: "5e68a861-d676-4ebc-8b1b-3adf489b7296"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 19:57:53.165452 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:53.165363    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e68a861-d676-4ebc-8b1b-3adf489b7296-maas-api-service-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:57:53.850568 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:53.850533    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a76d0719-nsdz4"
Jul 30 19:57:53.850752 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:53.850567    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a76d0719-nsdz4" event={"ID":"5e68a861-d676-4ebc-8b1b-3adf489b7296","Type":"ContainerDied","Data":"71209920a994fe9c51192000f89ab5a193c9041c05440f5c8dc41170c7e85e01"}
Jul 30 19:57:53.850752 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:53.850618    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="71209920a994fe9c51192000f89ab5a193c9041c05440f5c8dc41170c7e85e01"
Jul 30 19:57:55.273610 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:55.273562    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:57:55.847742 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:55.847712    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc"]
Jul 30 19:57:55.847982 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:55.847930    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc" podUID="3215b1b5-1f6b-4003-b2b1-c786acff3284" containerName="maas-api" containerID="cri-o://ded9c6f2fc533425b6f4c6ab316cc61c534a4a57a0877bbf88e458ed389440bc" gracePeriod=30
Jul 30 19:57:55.858214 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:55.858184    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:57:55.925238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:55.925206    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:57:55.997571 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:55.997508    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:57:56.039304 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:56.039261    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a76d0719-nsdz4"]
Jul 30 19:57:56.041722 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:56.041696    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a76d0719-nsdz4"]
Jul 30 19:57:56.114249 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:56.114169    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc"
Jul 30 19:57:56.159102 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:56.159067    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:57:56.194778 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:56.194749    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3215b1b5-1f6b-4003-b2b1-c786acff3284-maas-api-tls\") pod \"3215b1b5-1f6b-4003-b2b1-c786acff3284\" (UID: \"3215b1b5-1f6b-4003-b2b1-c786acff3284\") "
Jul 30 19:57:56.194949 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:56.194808    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zjrx\" (UniqueName: \"kubernetes.io/projected/3215b1b5-1f6b-4003-b2b1-c786acff3284-kube-api-access-9zjrx\") pod \"3215b1b5-1f6b-4003-b2b1-c786acff3284\" (UID: \"3215b1b5-1f6b-4003-b2b1-c786acff3284\") "
Jul 30 19:57:56.194949 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:56.194853    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3215b1b5-1f6b-4003-b2b1-c786acff3284-openshift-service-ca\") pod \"3215b1b5-1f6b-4003-b2b1-c786acff3284\" (UID: \"3215b1b5-1f6b-4003-b2b1-c786acff3284\") "
Jul 30 19:57:56.195293 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:56.195272    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3215b1b5-1f6b-4003-b2b1-c786acff3284-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3215b1b5-1f6b-4003-b2b1-c786acff3284" (UID: "3215b1b5-1f6b-4003-b2b1-c786acff3284"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 19:57:56.196863 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:56.196842    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3215b1b5-1f6b-4003-b2b1-c786acff3284-kube-api-access-9zjrx" (OuterVolumeSpecName: "kube-api-access-9zjrx") pod "3215b1b5-1f6b-4003-b2b1-c786acff3284" (UID: "3215b1b5-1f6b-4003-b2b1-c786acff3284"). InnerVolumeSpecName "kube-api-access-9zjrx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:57:56.196975 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:56.196878    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3215b1b5-1f6b-4003-b2b1-c786acff3284-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3215b1b5-1f6b-4003-b2b1-c786acff3284" (UID: "3215b1b5-1f6b-4003-b2b1-c786acff3284"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 19:57:56.295864 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:56.295833    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3215b1b5-1f6b-4003-b2b1-c786acff3284-maas-api-tls\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:57:56.295864 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:56.295862    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zjrx\" (UniqueName: \"kubernetes.io/projected/3215b1b5-1f6b-4003-b2b1-c786acff3284-kube-api-access-9zjrx\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:57:56.296296 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:56.295872    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3215b1b5-1f6b-4003-b2b1-c786acff3284-openshift-service-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:57:56.493345 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:56.493309    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e68a861-d676-4ebc-8b1b-3adf489b7296" path="/var/lib/kubelet/pods/5e68a861-d676-4ebc-8b1b-3adf489b7296/volumes"
Jul 30 19:57:56.870315 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:56.870218    2554 generic.go:356] "Generic (PLEG): container finished" podID="3215b1b5-1f6b-4003-b2b1-c786acff3284" containerID="ded9c6f2fc533425b6f4c6ab316cc61c534a4a57a0877bbf88e458ed389440bc" exitCode=0
Jul 30 19:57:56.870315 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:56.870260    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc" event={"ID":"3215b1b5-1f6b-4003-b2b1-c786acff3284","Type":"ContainerDied","Data":"ded9c6f2fc533425b6f4c6ab316cc61c534a4a57a0877bbf88e458ed389440bc"}
Jul 30 19:57:56.870315 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:56.870285    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc"
Jul 30 19:57:56.870315 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:56.870301    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc" event={"ID":"3215b1b5-1f6b-4003-b2b1-c786acff3284","Type":"ContainerDied","Data":"53699223a053765e8bc9df50ad7b374726df0bf9b53f6f5724aaf0af165f0c74"}
Jul 30 19:57:56.870657 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:56.870323    2554 scope.go:117] "RemoveContainer" containerID="ded9c6f2fc533425b6f4c6ab316cc61c534a4a57a0877bbf88e458ed389440bc"
Jul 30 19:57:56.880137 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:56.880118    2554 scope.go:117] "RemoveContainer" containerID="ded9c6f2fc533425b6f4c6ab316cc61c534a4a57a0877bbf88e458ed389440bc"
Jul 30 19:57:56.880421 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:57:56.880400    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ded9c6f2fc533425b6f4c6ab316cc61c534a4a57a0877bbf88e458ed389440bc\": container with ID starting with ded9c6f2fc533425b6f4c6ab316cc61c534a4a57a0877bbf88e458ed389440bc not found: ID does not exist" containerID="ded9c6f2fc533425b6f4c6ab316cc61c534a4a57a0877bbf88e458ed389440bc"
Jul 30 19:57:56.880497 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:56.880427    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ded9c6f2fc533425b6f4c6ab316cc61c534a4a57a0877bbf88e458ed389440bc"} err="failed to get container status \"ded9c6f2fc533425b6f4c6ab316cc61c534a4a57a0877bbf88e458ed389440bc\": rpc error: code = NotFound desc = could not find container \"ded9c6f2fc533425b6f4c6ab316cc61c534a4a57a0877bbf88e458ed389440bc\": container with ID starting with ded9c6f2fc533425b6f4c6ab316cc61c534a4a57a0877bbf88e458ed389440bc not found: ID does not exist"
Jul 30 19:57:56.891379 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:56.891352    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc"]
Jul 30 19:57:56.897712 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:56.897685    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a76d0719-6849669f9d-qjdnc"]
Jul 30 19:57:57.195980 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:57.195886    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"]
Jul 30 19:57:57.196156 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:57.196129    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k" podUID="9742486a-2ba8-487a-b00d-2e415649ff90" containerName="istio-proxy" containerID="cri-o://2f73999e8c7e79c7dc7c9de8e3e2a1dc4f8f010d5c1070db4321074940f06dff" gracePeriod=30
Jul 30 19:57:57.207024 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:57.206996    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:57:57.440199 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:57.440161    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:57:57.668179 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:57.668146    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:57:57.978470 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:57.978432    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:57:58.122531 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:58.122495    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:57:58.487177 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:57:58.487145    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3215b1b5-1f6b-4003-b2b1-c786acff3284" path="/var/lib/kubelet/pods/3215b1b5-1f6b-4003-b2b1-c786acff3284/volumes"
Jul 30 19:58:00.250126 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:00.250092    2554 patch_prober.go:28] interesting pod/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 19:58:00.250481 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:00.250145    2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k" podUID="9742486a-2ba8-487a-b00d-2e415649ff90" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 19:58:00.442765 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:00.442735    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:58:00.542310 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:00.542232    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:58:01.536151 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:01.536121    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r"]
Jul 30 19:58:01.536592 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:01.536577    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e68a861-d676-4ebc-8b1b-3adf489b7296" containerName="revoke-keys"
Jul 30 19:58:01.536637 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:01.536594    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e68a861-d676-4ebc-8b1b-3adf489b7296" containerName="revoke-keys"
Jul 30 19:58:01.536637 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:01.536611    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3215b1b5-1f6b-4003-b2b1-c786acff3284" containerName="maas-api"
Jul 30 19:58:01.536637 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:01.536617    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="3215b1b5-1f6b-4003-b2b1-c786acff3284" containerName="maas-api"
Jul 30 19:58:01.536733 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:01.536688    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e68a861-d676-4ebc-8b1b-3adf489b7296" containerName="revoke-keys"
Jul 30 19:58:01.536733 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:01.536698    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="3215b1b5-1f6b-4003-b2b1-c786acff3284" containerName="maas-api"
Jul 30 19:58:01.539729 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:01.539707    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r"
Jul 30 19:58:01.549512 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:01.549317    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-b83b31b4-serving-cert\""
Jul 30 19:58:01.555249 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:01.555218    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r"]
Jul 30 19:58:01.566111 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:01.566083    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:58:01.651155 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:01.651116    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51e706a1-3854-4b55-879b-68f448a911e5-maas-api-tls\") pod \"maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r\" (UID: \"51e706a1-3854-4b55-879b-68f448a911e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r"
Jul 30 19:58:01.651155 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:01.651154    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51e706a1-3854-4b55-879b-68f448a911e5-openshift-service-ca\") pod \"maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r\" (UID: \"51e706a1-3854-4b55-879b-68f448a911e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r"
Jul 30 19:58:01.651372 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:01.651241    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2l6p\" (UniqueName: \"kubernetes.io/projected/51e706a1-3854-4b55-879b-68f448a911e5-kube-api-access-m2l6p\") pod \"maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r\" (UID: \"51e706a1-3854-4b55-879b-68f448a911e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r"
Jul 30 19:58:01.752344 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:01.752294    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2l6p\" (UniqueName: \"kubernetes.io/projected/51e706a1-3854-4b55-879b-68f448a911e5-kube-api-access-m2l6p\") pod \"maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r\" (UID: \"51e706a1-3854-4b55-879b-68f448a911e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r"
Jul 30 19:58:01.752539 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:01.752392    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51e706a1-3854-4b55-879b-68f448a911e5-maas-api-tls\") pod \"maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r\" (UID: \"51e706a1-3854-4b55-879b-68f448a911e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r"
Jul 30 19:58:01.752539 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:01.752412    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51e706a1-3854-4b55-879b-68f448a911e5-openshift-service-ca\") pod \"maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r\" (UID: \"51e706a1-3854-4b55-879b-68f448a911e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r"
Jul 30 19:58:01.752539 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:58:01.752529    2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-serving-cert: secret "maas-api-e2e-derive-b83b31b4-serving-cert" not found
Jul 30 19:58:01.752650 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:58:01.752606    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51e706a1-3854-4b55-879b-68f448a911e5-maas-api-tls podName:51e706a1-3854-4b55-879b-68f448a911e5 nodeName:}" failed. No retries permitted until 2026-07-30 19:58:02.252589944 +0000 UTC m=+2010.217171003 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/51e706a1-3854-4b55-879b-68f448a911e5-maas-api-tls") pod "maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r" (UID: "51e706a1-3854-4b55-879b-68f448a911e5") : secret "maas-api-e2e-derive-b83b31b4-serving-cert" not found
Jul 30 19:58:01.753065 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:01.753049    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51e706a1-3854-4b55-879b-68f448a911e5-openshift-service-ca\") pod \"maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r\" (UID: \"51e706a1-3854-4b55-879b-68f448a911e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r"
Jul 30 19:58:01.760870 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:01.760847    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2l6p\" (UniqueName: \"kubernetes.io/projected/51e706a1-3854-4b55-879b-68f448a911e5-kube-api-access-m2l6p\") pod \"maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r\" (UID: \"51e706a1-3854-4b55-879b-68f448a911e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r"
Jul 30 19:58:02.040727 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.040699    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:58:02.141009 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.140921    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:58:02.205101 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.205068    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:58:02.257510 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.257479    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51e706a1-3854-4b55-879b-68f448a911e5-maas-api-tls\") pod \"maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r\" (UID: \"51e706a1-3854-4b55-879b-68f448a911e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r"
Jul 30 19:58:02.260413 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.260385    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51e706a1-3854-4b55-879b-68f448a911e5-maas-api-tls\") pod \"maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r\" (UID: \"51e706a1-3854-4b55-879b-68f448a911e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r"
Jul 30 19:58:02.456393 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.456362    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:58:02.460406 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.460385    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r"
Jul 30 19:58:02.560043 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.560011    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9742486a-2ba8-487a-b00d-2e415649ff90-istio-token\") pod \"9742486a-2ba8-487a-b00d-2e415649ff90\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") "
Jul 30 19:58:02.560504 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.560058    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-workload-certs\") pod \"9742486a-2ba8-487a-b00d-2e415649ff90\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") "
Jul 30 19:58:02.560504 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.560113    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-istio-envoy\") pod \"9742486a-2ba8-487a-b00d-2e415649ff90\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") "
Jul 30 19:58:02.560504 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.560147    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-istio-data\") pod \"9742486a-2ba8-487a-b00d-2e415649ff90\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") "
Jul 30 19:58:02.560504 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.560177    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-credential-socket\") pod \"9742486a-2ba8-487a-b00d-2e415649ff90\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") "
Jul 30 19:58:02.560504 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.560215    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9742486a-2ba8-487a-b00d-2e415649ff90-istiod-ca-cert\") pod \"9742486a-2ba8-487a-b00d-2e415649ff90\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") "
Jul 30 19:58:02.560504 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.560274    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4v9pc\" (UniqueName: \"kubernetes.io/projected/9742486a-2ba8-487a-b00d-2e415649ff90-kube-api-access-4v9pc\") pod \"9742486a-2ba8-487a-b00d-2e415649ff90\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") "
Jul 30 19:58:02.560504 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.560327    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9742486a-2ba8-487a-b00d-2e415649ff90-istio-podinfo\") pod \"9742486a-2ba8-487a-b00d-2e415649ff90\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") "
Jul 30 19:58:02.560504 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.560361    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-workload-socket\") pod \"9742486a-2ba8-487a-b00d-2e415649ff90\" (UID: \"9742486a-2ba8-487a-b00d-2e415649ff90\") "
Jul 30 19:58:02.560504 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.560383    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9742486a-2ba8-487a-b00d-2e415649ff90" (UID: "9742486a-2ba8-487a-b00d-2e415649ff90"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:58:02.561012 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.560526    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9742486a-2ba8-487a-b00d-2e415649ff90" (UID: "9742486a-2ba8-487a-b00d-2e415649ff90"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:58:02.561012 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.560756    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9742486a-2ba8-487a-b00d-2e415649ff90" (UID: "9742486a-2ba8-487a-b00d-2e415649ff90"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:58:02.561012 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.560999    2554 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-workload-certs\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:58:02.561186 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.561021    2554 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-credential-socket\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:58:02.561186 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.561037    2554 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-workload-socket\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:58:02.561186 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.561093    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9742486a-2ba8-487a-b00d-2e415649ff90-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9742486a-2ba8-487a-b00d-2e415649ff90" (UID: "9742486a-2ba8-487a-b00d-2e415649ff90"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 19:58:02.561712 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.561671    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-istio-data" (OuterVolumeSpecName: "istio-data") pod "9742486a-2ba8-487a-b00d-2e415649ff90" (UID: "9742486a-2ba8-487a-b00d-2e415649ff90"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:58:02.563641 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.563575    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9742486a-2ba8-487a-b00d-2e415649ff90" (UID: "9742486a-2ba8-487a-b00d-2e415649ff90"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 19:58:02.563641 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.563578    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9742486a-2ba8-487a-b00d-2e415649ff90-istio-token" (OuterVolumeSpecName: "istio-token") pod "9742486a-2ba8-487a-b00d-2e415649ff90" (UID: "9742486a-2ba8-487a-b00d-2e415649ff90"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:58:02.563641 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.563622    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9742486a-2ba8-487a-b00d-2e415649ff90-kube-api-access-4v9pc" (OuterVolumeSpecName: "kube-api-access-4v9pc") pod "9742486a-2ba8-487a-b00d-2e415649ff90" (UID: "9742486a-2ba8-487a-b00d-2e415649ff90"). InnerVolumeSpecName "kube-api-access-4v9pc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:58:02.564905 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.564877    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9742486a-2ba8-487a-b00d-2e415649ff90-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9742486a-2ba8-487a-b00d-2e415649ff90" (UID: "9742486a-2ba8-487a-b00d-2e415649ff90"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 19:58:02.600497 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.600471    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r"]
Jul 30 19:58:02.602385 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:58:02.602358    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51e706a1_3854_4b55_879b_68f448a911e5.slice/crio-a6a61f3b5b9f489ce34c95d9ed4d1e14ed1fb38814cab559d7d4f102400c2df3 WatchSource:0}: Error finding container a6a61f3b5b9f489ce34c95d9ed4d1e14ed1fb38814cab559d7d4f102400c2df3: Status 404 returned error can't find the container with id a6a61f3b5b9f489ce34c95d9ed4d1e14ed1fb38814cab559d7d4f102400c2df3
Jul 30 19:58:02.661525 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.661499    2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9742486a-2ba8-487a-b00d-2e415649ff90-istio-token\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:58:02.661525 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.661522    2554 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-istio-envoy\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:58:02.661525 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.661530    2554 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9742486a-2ba8-487a-b00d-2e415649ff90-istio-data\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:58:02.661726 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.661540    2554 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9742486a-2ba8-487a-b00d-2e415649ff90-istiod-ca-cert\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:58:02.661726 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.661548    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4v9pc\" (UniqueName: \"kubernetes.io/projected/9742486a-2ba8-487a-b00d-2e415649ff90-kube-api-access-4v9pc\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:58:02.661726 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.661558    2554 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9742486a-2ba8-487a-b00d-2e415649ff90-istio-podinfo\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:58:02.909641 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.909545    2554 generic.go:356] "Generic (PLEG): container finished" podID="9742486a-2ba8-487a-b00d-2e415649ff90" containerID="2f73999e8c7e79c7dc7c9de8e3e2a1dc4f8f010d5c1070db4321074940f06dff" exitCode=0
Jul 30 19:58:02.909641 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.909600    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"
Jul 30 19:58:02.909866 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.909633    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k" event={"ID":"9742486a-2ba8-487a-b00d-2e415649ff90","Type":"ContainerDied","Data":"2f73999e8c7e79c7dc7c9de8e3e2a1dc4f8f010d5c1070db4321074940f06dff"}
Jul 30 19:58:02.909866 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.909704    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k" event={"ID":"9742486a-2ba8-487a-b00d-2e415649ff90","Type":"ContainerDied","Data":"329ec1823a3a51d1480dcd7956c6e399d331cc833f9533fc6a91363a2bb4234b"}
Jul 30 19:58:02.909866 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.909735    2554 scope.go:117] "RemoveContainer" containerID="2f73999e8c7e79c7dc7c9de8e3e2a1dc4f8f010d5c1070db4321074940f06dff"
Jul 30 19:58:02.911163 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.911138    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r" event={"ID":"51e706a1-3854-4b55-879b-68f448a911e5","Type":"ContainerStarted","Data":"a6a61f3b5b9f489ce34c95d9ed4d1e14ed1fb38814cab559d7d4f102400c2df3"}
Jul 30 19:58:02.920444 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.920426    2554 scope.go:117] "RemoveContainer" containerID="84eeec88782c8eef8c77dca30b4b1be514738d5a90b7a46769455a03396dcd6f"
Jul 30 19:58:02.928926 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.928909    2554 scope.go:117] "RemoveContainer" containerID="2f73999e8c7e79c7dc7c9de8e3e2a1dc4f8f010d5c1070db4321074940f06dff"
Jul 30 19:58:02.929294 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:58:02.929270    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f73999e8c7e79c7dc7c9de8e3e2a1dc4f8f010d5c1070db4321074940f06dff\": container with ID starting with 2f73999e8c7e79c7dc7c9de8e3e2a1dc4f8f010d5c1070db4321074940f06dff not found: ID does not exist" containerID="2f73999e8c7e79c7dc7c9de8e3e2a1dc4f8f010d5c1070db4321074940f06dff"
Jul 30 19:58:02.929294 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.929304    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f73999e8c7e79c7dc7c9de8e3e2a1dc4f8f010d5c1070db4321074940f06dff"} err="failed to get container status \"2f73999e8c7e79c7dc7c9de8e3e2a1dc4f8f010d5c1070db4321074940f06dff\": rpc error: code = NotFound desc = could not find container \"2f73999e8c7e79c7dc7c9de8e3e2a1dc4f8f010d5c1070db4321074940f06dff\": container with ID starting with 2f73999e8c7e79c7dc7c9de8e3e2a1dc4f8f010d5c1070db4321074940f06dff not found: ID does not exist"
Jul 30 19:58:02.929582 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.929325    2554 scope.go:117] "RemoveContainer" containerID="84eeec88782c8eef8c77dca30b4b1be514738d5a90b7a46769455a03396dcd6f"
Jul 30 19:58:02.929883 ip-10-0-134-180 kubenswrapper[2554]: E0730 19:58:02.929813    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84eeec88782c8eef8c77dca30b4b1be514738d5a90b7a46769455a03396dcd6f\": container with ID starting with 84eeec88782c8eef8c77dca30b4b1be514738d5a90b7a46769455a03396dcd6f not found: ID does not exist" containerID="84eeec88782c8eef8c77dca30b4b1be514738d5a90b7a46769455a03396dcd6f"
Jul 30 19:58:02.929997 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.929897    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84eeec88782c8eef8c77dca30b4b1be514738d5a90b7a46769455a03396dcd6f"} err="failed to get container status \"84eeec88782c8eef8c77dca30b4b1be514738d5a90b7a46769455a03396dcd6f\": rpc error: code = NotFound desc = could not find container \"84eeec88782c8eef8c77dca30b4b1be514738d5a90b7a46769455a03396dcd6f\": container with ID starting with 84eeec88782c8eef8c77dca30b4b1be514738d5a90b7a46769455a03396dcd6f not found: ID does not exist"
Jul 30 19:58:02.932238 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.932219    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"]
Jul 30 19:58:02.936427 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:02.936397    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a76d0719-openshift-default-cdc4b7dc8-pps4k"]
Jul 30 19:58:03.921007 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:03.920972    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r" event={"ID":"51e706a1-3854-4b55-879b-68f448a911e5","Type":"ContainerStarted","Data":"832e3b5343209f9c9a1f3dfb8bb1b22287c6f404ec9b5e64a20dca4bf5b6fe88"}
Jul 30 19:58:03.921499 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:03.921181    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r"
Jul 30 19:58:03.937537 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:03.937487    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r" podStartSLOduration=2.573309773 podStartE2EDuration="2.937470388s" totalImagesPullingTime="364.160615ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 19:58:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 19:58:03.93639774 +0000 UTC m=+2011.900978821" watchObservedRunningTime="2026-07-30 19:58:03.937470388 +0000 UTC m=+2011.902051469"
Jul 30 19:58:04.487304 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:04.487269    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9742486a-2ba8-487a-b00d-2e415649ff90" path="/var/lib/kubelet/pods/9742486a-2ba8-487a-b00d-2e415649ff90/volumes"
Jul 30 19:58:09.932389 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:09.932356    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r"
Jul 30 19:58:25.333783 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:25.333746    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b83b31b4-f26d6"]
Jul 30 19:58:25.334219 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:25.334176    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9742486a-2ba8-487a-b00d-2e415649ff90" containerName="istio-proxy"
Jul 30 19:58:25.334219 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:25.334189    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="9742486a-2ba8-487a-b00d-2e415649ff90" containerName="istio-proxy"
Jul 30 19:58:25.334292 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:25.334264    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="9742486a-2ba8-487a-b00d-2e415649ff90" containerName="istio-proxy"
Jul 30 19:58:25.337415 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:25.337397    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b83b31b4-f26d6"
Jul 30 19:58:25.339851 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:25.339822    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7t7kk\""
Jul 30 19:58:25.346836 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:25.346800    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b83b31b4-f26d6"]
Jul 30 19:58:25.481133 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:25.481088    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6e93d4c-956e-4a7e-a3d4-e62224c0baa6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-b83b31b4-f26d6\" (UID: \"f6e93d4c-956e-4a7e-a3d4-e62224c0baa6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b83b31b4-f26d6"
Jul 30 19:58:25.582621 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:25.582581    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6e93d4c-956e-4a7e-a3d4-e62224c0baa6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-b83b31b4-f26d6\" (UID: \"f6e93d4c-956e-4a7e-a3d4-e62224c0baa6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b83b31b4-f26d6"
Jul 30 19:58:25.583260 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:25.583239    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6e93d4c-956e-4a7e-a3d4-e62224c0baa6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-b83b31b4-f26d6\" (UID: \"f6e93d4c-956e-4a7e-a3d4-e62224c0baa6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b83b31b4-f26d6"
Jul 30 19:58:25.658890 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:25.658808    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b83b31b4-f26d6"
Jul 30 19:58:25.795923 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:25.795891    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b83b31b4-f26d6"]
Jul 30 19:58:25.797434 ip-10-0-134-180 kubenswrapper[2554]: W0730 19:58:25.797398    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6e93d4c_956e_4a7e_a3d4_e62224c0baa6.slice/crio-8dca030a4aff4c7cf0aabb08948fce22d02768922776a10791e5dbcf0508ddc2 WatchSource:0}: Error finding container 8dca030a4aff4c7cf0aabb08948fce22d02768922776a10791e5dbcf0508ddc2: Status 404 returned error can't find the container with id 8dca030a4aff4c7cf0aabb08948fce22d02768922776a10791e5dbcf0508ddc2
Jul 30 19:58:26.059851 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:26.059812    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b83b31b4-f26d6" event={"ID":"f6e93d4c-956e-4a7e-a3d4-e62224c0baa6","Type":"ContainerStarted","Data":"8dca030a4aff4c7cf0aabb08948fce22d02768922776a10791e5dbcf0508ddc2"}
Jul 30 19:58:27.067445 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:27.067412    2554 generic.go:356] "Generic (PLEG): container finished" podID="f6e93d4c-956e-4a7e-a3d4-e62224c0baa6" containerID="29cf6ae5095e65ad3ccfe6f7b1d34bfb59a745c7822864fd734de2d1cde56d0e" exitCode=0
Jul 30 19:58:27.067834 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:27.067504    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b83b31b4-f26d6" event={"ID":"f6e93d4c-956e-4a7e-a3d4-e62224c0baa6","Type":"ContainerDied","Data":"29cf6ae5095e65ad3ccfe6f7b1d34bfb59a745c7822864fd734de2d1cde56d0e"}
Jul 30 19:58:28.207457 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:28.207434    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b83b31b4-f26d6"
Jul 30 19:58:28.210554 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:28.210532    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6e93d4c-956e-4a7e-a3d4-e62224c0baa6-maas-api-service-ca\") pod \"f6e93d4c-956e-4a7e-a3d4-e62224c0baa6\" (UID: \"f6e93d4c-956e-4a7e-a3d4-e62224c0baa6\") "
Jul 30 19:58:28.210881 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:28.210858    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6e93d4c-956e-4a7e-a3d4-e62224c0baa6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f6e93d4c-956e-4a7e-a3d4-e62224c0baa6" (UID: "f6e93d4c-956e-4a7e-a3d4-e62224c0baa6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 19:58:28.311326 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:28.311287    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6e93d4c-956e-4a7e-a3d4-e62224c0baa6-maas-api-service-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:58:29.080432 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:29.080402    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b83b31b4-f26d6"
Jul 30 19:58:29.080432 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:29.080409    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b83b31b4-f26d6" event={"ID":"f6e93d4c-956e-4a7e-a3d4-e62224c0baa6","Type":"ContainerDied","Data":"8dca030a4aff4c7cf0aabb08948fce22d02768922776a10791e5dbcf0508ddc2"}
Jul 30 19:58:29.080651 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:29.080461    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8dca030a4aff4c7cf0aabb08948fce22d02768922776a10791e5dbcf0508ddc2"
Jul 30 19:58:31.965670 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:31.965634    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r"]
Jul 30 19:58:31.966231 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:31.965875    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r" podUID="51e706a1-3854-4b55-879b-68f448a911e5" containerName="maas-api" containerID="cri-o://832e3b5343209f9c9a1f3dfb8bb1b22287c6f404ec9b5e64a20dca4bf5b6fe88" gracePeriod=30
Jul 30 19:58:31.987888 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:31.987859    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:58:32.049244 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:32.049193    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:58:32.103011 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:32.102979    2554 generic.go:356] "Generic (PLEG): container finished" podID="51e706a1-3854-4b55-879b-68f448a911e5" containerID="832e3b5343209f9c9a1f3dfb8bb1b22287c6f404ec9b5e64a20dca4bf5b6fe88" exitCode=0
Jul 30 19:58:32.103189 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:32.103057    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r" event={"ID":"51e706a1-3854-4b55-879b-68f448a911e5","Type":"ContainerDied","Data":"832e3b5343209f9c9a1f3dfb8bb1b22287c6f404ec9b5e64a20dca4bf5b6fe88"}
Jul 30 19:58:32.144381 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:32.144349    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b83b31b4-f26d6"]
Jul 30 19:58:32.145162 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:32.145138    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:58:32.147980 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:32.147411    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b83b31b4-f26d6"]
Jul 30 19:58:32.197783 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:32.197739    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:58:32.229785 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:32.229722    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r"
Jul 30 19:58:32.248577 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:32.248555    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51e706a1-3854-4b55-879b-68f448a911e5-openshift-service-ca\") pod \"51e706a1-3854-4b55-879b-68f448a911e5\" (UID: \"51e706a1-3854-4b55-879b-68f448a911e5\") "
Jul 30 19:58:32.248694 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:32.248594    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51e706a1-3854-4b55-879b-68f448a911e5-maas-api-tls\") pod \"51e706a1-3854-4b55-879b-68f448a911e5\" (UID: \"51e706a1-3854-4b55-879b-68f448a911e5\") "
Jul 30 19:58:32.248742 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:32.248707    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2l6p\" (UniqueName: \"kubernetes.io/projected/51e706a1-3854-4b55-879b-68f448a911e5-kube-api-access-m2l6p\") pod \"51e706a1-3854-4b55-879b-68f448a911e5\" (UID: \"51e706a1-3854-4b55-879b-68f448a911e5\") "
Jul 30 19:58:32.248984 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:32.248942    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51e706a1-3854-4b55-879b-68f448a911e5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "51e706a1-3854-4b55-879b-68f448a911e5" (UID: "51e706a1-3854-4b55-879b-68f448a911e5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 19:58:32.251242 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:32.251214    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51e706a1-3854-4b55-879b-68f448a911e5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "51e706a1-3854-4b55-879b-68f448a911e5" (UID: "51e706a1-3854-4b55-879b-68f448a911e5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 19:58:32.251736 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:32.251713    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51e706a1-3854-4b55-879b-68f448a911e5-kube-api-access-m2l6p" (OuterVolumeSpecName: "kube-api-access-m2l6p") pod "51e706a1-3854-4b55-879b-68f448a911e5" (UID: "51e706a1-3854-4b55-879b-68f448a911e5"). InnerVolumeSpecName "kube-api-access-m2l6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 19:58:32.298909 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:32.298877    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:58:32.349663 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:32.349632    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2l6p\" (UniqueName: \"kubernetes.io/projected/51e706a1-3854-4b55-879b-68f448a911e5-kube-api-access-m2l6p\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:58:32.349663 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:32.349656    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51e706a1-3854-4b55-879b-68f448a911e5-openshift-service-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:58:32.349663 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:32.349666    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51e706a1-3854-4b55-879b-68f448a911e5-maas-api-tls\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 19:58:32.488757 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:32.488677    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6e93d4c-956e-4a7e-a3d4-e62224c0baa6" path="/var/lib/kubelet/pods/f6e93d4c-956e-4a7e-a3d4-e62224c0baa6/volumes"
Jul 30 19:58:32.529116 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:32.529089    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:58:33.012640 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:33.012602    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:58:33.104307 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:33.104272    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:58:33.110585 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:33.110553    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r" event={"ID":"51e706a1-3854-4b55-879b-68f448a911e5","Type":"ContainerDied","Data":"a6a61f3b5b9f489ce34c95d9ed4d1e14ed1fb38814cab559d7d4f102400c2df3"}
Jul 30 19:58:33.110759 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:33.110619    2554 scope.go:117] "RemoveContainer" containerID="832e3b5343209f9c9a1f3dfb8bb1b22287c6f404ec9b5e64a20dca4bf5b6fe88"
Jul 30 19:58:33.110759 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:33.110559    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r"
Jul 30 19:58:33.135090 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:33.135059    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r"]
Jul 30 19:58:33.140756 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:33.140731    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b83b31b4-cbdf94997-hf66r"]
Jul 30 19:58:33.204993 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:33.204949    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:58:34.492340 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:34.492305    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51e706a1-3854-4b55-879b-68f448a911e5" path="/var/lib/kubelet/pods/51e706a1-3854-4b55-879b-68f448a911e5/volumes"
Jul 30 19:58:55.140886 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:58:55.140805    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:59:00.824897 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:59:00.824861    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:59:00.931813 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:59:00.931775    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:59:01.026408 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:59:01.026372    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:59:01.127469 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:59:01.127382    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:59:01.593297 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:59:01.593263    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:59:27.521143 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:59:27.521106    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:59:32.664899 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:59:32.664866    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/1.log"
Jul 30 19:59:32.669474 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:59:32.669447    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 19:59:32.672109 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:59:32.672089    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/1.log"
Jul 30 19:59:32.676649 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:59:32.676632    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 19:59:33.641007 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:59:33.640974    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:59:33.743235 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:59:33.743199    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:59:33.843439 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:59:33.843405    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:59:49.747217 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:59:49.747179    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:59:55.975407 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:59:55.975370    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:59:56.042324 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:59:56.042292    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:59:56.136826 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:59:56.136794    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 19:59:56.238324 ip-10-0-134-180 kubenswrapper[2554]: I0730 19:59:56.238238    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:00:00.162237 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:00.162200    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757360-kptpq"]
Jul 30 20:00:00.162707 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:00.162606    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51e706a1-3854-4b55-879b-68f448a911e5" containerName="maas-api"
Jul 30 20:00:00.162707 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:00.162619    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="51e706a1-3854-4b55-879b-68f448a911e5" containerName="maas-api"
Jul 30 20:00:00.162707 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:00.162647    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6e93d4c-956e-4a7e-a3d4-e62224c0baa6" containerName="revoke-keys"
Jul 30 20:00:00.162707 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:00.162653    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6e93d4c-956e-4a7e-a3d4-e62224c0baa6" containerName="revoke-keys"
Jul 30 20:00:00.162707 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:00.162664    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9742486a-2ba8-487a-b00d-2e415649ff90" containerName="istio-proxy"
Jul 30 20:00:00.162707 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:00.162670    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="9742486a-2ba8-487a-b00d-2e415649ff90" containerName="istio-proxy"
Jul 30 20:00:00.162899 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:00.162728    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6e93d4c-956e-4a7e-a3d4-e62224c0baa6" containerName="revoke-keys"
Jul 30 20:00:00.162899 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:00.162737    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="9742486a-2ba8-487a-b00d-2e415649ff90" containerName="istio-proxy"
Jul 30 20:00:00.162899 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:00.162744    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="51e706a1-3854-4b55-879b-68f448a911e5" containerName="maas-api"
Jul 30 20:00:00.165790 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:00.165773    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757360-kptpq"
Jul 30 20:00:00.168647 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:00.168625    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7t7kk\""
Jul 30 20:00:00.176215 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:00.176193    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757360-kptpq"]
Jul 30 20:00:00.176539 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:00.176523    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757360-kptpq"
Jul 30 20:00:00.315366 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:00.315338    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757360-kptpq"]
Jul 30 20:00:00.317514 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:00:00.317481    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc61d13c_d148_4670_a971_17df143d3509.slice/crio-c0629fd7ed7b5b575ba347150d7e4addc89a9f0104795ae8c5b70177b16efe1a WatchSource:0}: Error finding container c0629fd7ed7b5b575ba347150d7e4addc89a9f0104795ae8c5b70177b16efe1a: Status 404 returned error can't find the container with id c0629fd7ed7b5b575ba347150d7e4addc89a9f0104795ae8c5b70177b16efe1a
Jul 30 20:00:00.630260 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:00.630229    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757360-kptpq" event={"ID":"dc61d13c-d148-4670-a971-17df143d3509","Type":"ContainerStarted","Data":"4826623da2aebe0ba3515a47d3a3e3a1c6f22152cb4ae89388b4331b82675cab"}
Jul 30 20:00:00.630379 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:00.630280    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757360-kptpq" event={"ID":"dc61d13c-d148-4670-a971-17df143d3509","Type":"ContainerStarted","Data":"c0629fd7ed7b5b575ba347150d7e4addc89a9f0104795ae8c5b70177b16efe1a"}
Jul 30 20:00:00.647776 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:00.647728    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757360-kptpq" podStartSLOduration=0.647713453 podStartE2EDuration="647.713453ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:00:00.646465 +0000 UTC m=+2128.611046082" watchObservedRunningTime="2026-07-30 20:00:00.647713453 +0000 UTC m=+2128.612294533"
Jul 30 20:00:01.638673 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:01.638639    2554 generic.go:356] "Generic (PLEG): container finished" podID="dc61d13c-d148-4670-a971-17df143d3509" containerID="4826623da2aebe0ba3515a47d3a3e3a1c6f22152cb4ae89388b4331b82675cab" exitCode=0
Jul 30 20:00:01.639129 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:01.638731    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757360-kptpq" event={"ID":"dc61d13c-d148-4670-a971-17df143d3509","Type":"ContainerDied","Data":"4826623da2aebe0ba3515a47d3a3e3a1c6f22152cb4ae89388b4331b82675cab"}
Jul 30 20:00:02.781550 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:02.781524    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757360-kptpq"
Jul 30 20:00:03.652000 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:03.651966    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757360-kptpq"
Jul 30 20:00:03.652000 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:03.651989    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757360-kptpq" event={"ID":"dc61d13c-d148-4670-a971-17df143d3509","Type":"ContainerDied","Data":"c0629fd7ed7b5b575ba347150d7e4addc89a9f0104795ae8c5b70177b16efe1a"}
Jul 30 20:00:03.652215 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:03.652040    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c0629fd7ed7b5b575ba347150d7e4addc89a9f0104795ae8c5b70177b16efe1a"
Jul 30 20:00:06.744472 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:06.744436    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:00:17.777632 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:17.777600    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:00:24.006167 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:24.006130    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:00:24.110400 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:24.110368    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:00:24.228883 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:24.228842    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:00:24.331741 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:24.331654    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:00:25.109481 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:25.109434    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:00:25.214401 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:25.214357    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:00:25.330340 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:25.330300    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:00:26.902200 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:26.902167    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:00:34.567117 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:34.567079    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:00:34.631102 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:34.631065    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:00:35.301989 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:35.301933    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:00:41.373271 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:41.373239    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:00:41.429497 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:41.429464    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:00:41.532329 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:41.532294    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:00:41.872409 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:41.872369    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:00:42.503080 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:42.503048    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:00:44.724532 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:44.724500    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:00:44.816017 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:44.815985    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:00:45.792524 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:45.792483    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6"]
Jul 30 20:00:45.793118 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:45.793100    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc61d13c-d148-4670-a971-17df143d3509" containerName="cleanup"
Jul 30 20:00:45.793170 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:45.793121    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc61d13c-d148-4670-a971-17df143d3509" containerName="cleanup"
Jul 30 20:00:45.793238 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:45.793227    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc61d13c-d148-4670-a971-17df143d3509" containerName="cleanup"
Jul 30 20:00:45.796722 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:45.796701    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6"
Jul 30 20:00:45.799245 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:45.799222    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-361f6a62-serving-cert\""
Jul 30 20:00:45.808130 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:45.808106    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6"]
Jul 30 20:00:45.887789 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:45.887759    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7be0d75f-3265-4012-8deb-4505b5332f1e-openshift-service-ca\") pod \"maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6\" (UID: \"7be0d75f-3265-4012-8deb-4505b5332f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6"
Jul 30 20:00:45.888027 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:45.887801    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46vl4\" (UniqueName: \"kubernetes.io/projected/7be0d75f-3265-4012-8deb-4505b5332f1e-kube-api-access-46vl4\") pod \"maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6\" (UID: \"7be0d75f-3265-4012-8deb-4505b5332f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6"
Jul 30 20:00:45.888027 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:45.887987    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7be0d75f-3265-4012-8deb-4505b5332f1e-maas-api-tls\") pod \"maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6\" (UID: \"7be0d75f-3265-4012-8deb-4505b5332f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6"
Jul 30 20:00:45.966285 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:45.966249    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:00:45.989005 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:45.988947    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46vl4\" (UniqueName: \"kubernetes.io/projected/7be0d75f-3265-4012-8deb-4505b5332f1e-kube-api-access-46vl4\") pod \"maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6\" (UID: \"7be0d75f-3265-4012-8deb-4505b5332f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6"
Jul 30 20:00:45.989183 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:45.989130    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7be0d75f-3265-4012-8deb-4505b5332f1e-maas-api-tls\") pod \"maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6\" (UID: \"7be0d75f-3265-4012-8deb-4505b5332f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6"
Jul 30 20:00:45.989254 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:45.989194    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7be0d75f-3265-4012-8deb-4505b5332f1e-openshift-service-ca\") pod \"maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6\" (UID: \"7be0d75f-3265-4012-8deb-4505b5332f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6"
Jul 30 20:00:45.989829 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:45.989806    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7be0d75f-3265-4012-8deb-4505b5332f1e-openshift-service-ca\") pod \"maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6\" (UID: \"7be0d75f-3265-4012-8deb-4505b5332f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6"
Jul 30 20:00:45.991912 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:45.991890    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7be0d75f-3265-4012-8deb-4505b5332f1e-maas-api-tls\") pod \"maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6\" (UID: \"7be0d75f-3265-4012-8deb-4505b5332f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6"
Jul 30 20:00:46.000059 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:46.000037    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46vl4\" (UniqueName: \"kubernetes.io/projected/7be0d75f-3265-4012-8deb-4505b5332f1e-kube-api-access-46vl4\") pod \"maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6\" (UID: \"7be0d75f-3265-4012-8deb-4505b5332f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6"
Jul 30 20:00:46.061386 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:46.061266    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:00:46.112416 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:46.112386    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6"
Jul 30 20:00:46.256872 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:46.256843    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6"]
Jul 30 20:00:46.258011 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:00:46.257986    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7be0d75f_3265_4012_8deb_4505b5332f1e.slice/crio-5bca39ff49dbc0c6efb4df8a4eba0f02e3e757e4f87952dc965633b2573524c8 WatchSource:0}: Error finding container 5bca39ff49dbc0c6efb4df8a4eba0f02e3e757e4f87952dc965633b2573524c8: Status 404 returned error can't find the container with id 5bca39ff49dbc0c6efb4df8a4eba0f02e3e757e4f87952dc965633b2573524c8
Jul 30 20:00:46.340387 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:46.340283    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:00:46.439544 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:46.439513    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:00:46.474908 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:46.474876    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j"]
Jul 30 20:00:46.479785 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:46.479764    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j"
Jul 30 20:00:46.482979 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:46.482923    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-361f6a62\""
Jul 30 20:00:46.483100 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:46.482980    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-361f6a62-dockercfg-nsn2d\""
Jul 30 20:00:46.493196 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:46.493115    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j"]
Jul 30 20:00:46.493584 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:46.493560    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bf11f5b-9367-4d9c-95dc-0b00babac51a-plugins-config-volume\") pod \"payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j\" (UID: \"6bf11f5b-9367-4d9c-95dc-0b00babac51a\") " pod="openshift-ingress/payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j"
Jul 30 20:00:46.493671 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:46.493643    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm9dn\" (UniqueName: \"kubernetes.io/projected/6bf11f5b-9367-4d9c-95dc-0b00babac51a-kube-api-access-gm9dn\") pod \"payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j\" (UID: \"6bf11f5b-9367-4d9c-95dc-0b00babac51a\") " pod="openshift-ingress/payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j"
Jul 30 20:00:46.594413 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:46.594336    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm9dn\" (UniqueName: \"kubernetes.io/projected/6bf11f5b-9367-4d9c-95dc-0b00babac51a-kube-api-access-gm9dn\") pod \"payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j\" (UID: \"6bf11f5b-9367-4d9c-95dc-0b00babac51a\") " pod="openshift-ingress/payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j"
Jul 30 20:00:46.594413 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:46.594413    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bf11f5b-9367-4d9c-95dc-0b00babac51a-plugins-config-volume\") pod \"payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j\" (UID: \"6bf11f5b-9367-4d9c-95dc-0b00babac51a\") " pod="openshift-ingress/payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j"
Jul 30 20:00:46.595067 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:46.595048    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bf11f5b-9367-4d9c-95dc-0b00babac51a-plugins-config-volume\") pod \"payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j\" (UID: \"6bf11f5b-9367-4d9c-95dc-0b00babac51a\") " pod="openshift-ingress/payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j"
Jul 30 20:00:46.603388 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:46.603358    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm9dn\" (UniqueName: \"kubernetes.io/projected/6bf11f5b-9367-4d9c-95dc-0b00babac51a-kube-api-access-gm9dn\") pod \"payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j\" (UID: \"6bf11f5b-9367-4d9c-95dc-0b00babac51a\") " pod="openshift-ingress/payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j"
Jul 30 20:00:46.792623 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:46.792587    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j"
Jul 30 20:00:46.911708 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:46.911669    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6" event={"ID":"7be0d75f-3265-4012-8deb-4505b5332f1e","Type":"ContainerStarted","Data":"5bca39ff49dbc0c6efb4df8a4eba0f02e3e757e4f87952dc965633b2573524c8"}
Jul 30 20:00:46.944685 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:46.940973    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j"]
Jul 30 20:00:47.921938 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:47.921896    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6" event={"ID":"7be0d75f-3265-4012-8deb-4505b5332f1e","Type":"ContainerStarted","Data":"ab155de717eb57f1fb27ad30970c0f34d6a2deae9d7b89243bc0d9841d57ff34"}
Jul 30 20:00:47.922441 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:47.922122    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6"
Jul 30 20:00:47.923520 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:47.923491    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j" event={"ID":"6bf11f5b-9367-4d9c-95dc-0b00babac51a","Type":"ContainerStarted","Data":"5e781171da34d66e61930b1d66d454c79859fd2e21ec28152bd6346ea25493fd"}
Jul 30 20:00:47.941571 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:47.941515    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6" podStartSLOduration=2.154713639 podStartE2EDuration="2.941495442s" totalImagesPullingTime="786.781803ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:00:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:00:47.937390618 +0000 UTC m=+2175.901971700" watchObservedRunningTime="2026-07-30 20:00:47.941495442 +0000 UTC m=+2175.906076523"
Jul 30 20:00:48.931507 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:48.931392    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j" event={"ID":"6bf11f5b-9367-4d9c-95dc-0b00babac51a","Type":"ContainerStarted","Data":"1d5020a7f43f6aa42a1d207b81d8c8fe51253ab12b85269c8fe64c6ae70b3bef"}
Jul 30 20:00:48.931997 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:48.931554    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j"
Jul 30 20:00:48.952300 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:48.952234    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j" podStartSLOduration=1.270164182 podStartE2EDuration="2.952212129s" totalImagesPullingTime="1.682047947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:00:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:00:48.948096257 +0000 UTC m=+2176.912677338" watchObservedRunningTime="2026-07-30 20:00:48.952212129 +0000 UTC m=+2176.916793212"
Jul 30 20:00:53.937092 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:53.937062    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6"
Jul 30 20:00:59.939982 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:00:59.939919    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j"
Jul 30 20:01:09.727603 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:09.727566    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:01:10.419517 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:10.419484    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:01:10.519468 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:10.519429    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:01:10.871408 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:10.871369    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:01:13.712743 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:13.712709    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:01:15.262262 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:15.262204    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:01:15.313086 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:15.313051    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:01:16.443438 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:16.443399    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:01:16.543857 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:16.543815    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:01:27.211029 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:27.210484    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6"]
Jul 30 20:01:27.211029 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:27.210751    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6" podUID="7be0d75f-3265-4012-8deb-4505b5332f1e" containerName="maas-api" containerID="cri-o://ab155de717eb57f1fb27ad30970c0f34d6a2deae9d7b89243bc0d9841d57ff34" gracePeriod=30
Jul 30 20:01:27.234178 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:27.234134    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:01:27.266934 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:27.266904    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j"]
Jul 30 20:01:27.267144 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:27.267117    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j" podUID="6bf11f5b-9367-4d9c-95dc-0b00babac51a" containerName="payload-processing" containerID="cri-o://1d5020a7f43f6aa42a1d207b81d8c8fe51253ab12b85269c8fe64c6ae70b3bef" gracePeriod=30
Jul 30 20:01:27.306514 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:27.306483    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:01:27.412733 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:27.412663    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:01:27.470387 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:27.470295    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:01:27.528385 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:27.528358    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6"
Jul 30 20:01:27.551608 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:27.551575    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j"
Jul 30 20:01:27.580896 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:27.580868    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gm9dn\" (UniqueName: \"kubernetes.io/projected/6bf11f5b-9367-4d9c-95dc-0b00babac51a-kube-api-access-gm9dn\") pod \"6bf11f5b-9367-4d9c-95dc-0b00babac51a\" (UID: \"6bf11f5b-9367-4d9c-95dc-0b00babac51a\") "
Jul 30 20:01:27.581081 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:27.580904    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7be0d75f-3265-4012-8deb-4505b5332f1e-maas-api-tls\") pod \"7be0d75f-3265-4012-8deb-4505b5332f1e\" (UID: \"7be0d75f-3265-4012-8deb-4505b5332f1e\") "
Jul 30 20:01:27.581081 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:27.580943    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46vl4\" (UniqueName: \"kubernetes.io/projected/7be0d75f-3265-4012-8deb-4505b5332f1e-kube-api-access-46vl4\") pod \"7be0d75f-3265-4012-8deb-4505b5332f1e\" (UID: \"7be0d75f-3265-4012-8deb-4505b5332f1e\") "
Jul 30 20:01:27.581081 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:27.580993    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7be0d75f-3265-4012-8deb-4505b5332f1e-openshift-service-ca\") pod \"7be0d75f-3265-4012-8deb-4505b5332f1e\" (UID: \"7be0d75f-3265-4012-8deb-4505b5332f1e\") "
Jul 30 20:01:27.581081 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:27.581027    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bf11f5b-9367-4d9c-95dc-0b00babac51a-plugins-config-volume\") pod \"6bf11f5b-9367-4d9c-95dc-0b00babac51a\" (UID: \"6bf11f5b-9367-4d9c-95dc-0b00babac51a\") "
Jul 30 20:01:27.581429 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:27.581395    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7be0d75f-3265-4012-8deb-4505b5332f1e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7be0d75f-3265-4012-8deb-4505b5332f1e" (UID: "7be0d75f-3265-4012-8deb-4505b5332f1e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:01:27.581568 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:27.581480    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bf11f5b-9367-4d9c-95dc-0b00babac51a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6bf11f5b-9367-4d9c-95dc-0b00babac51a" (UID: "6bf11f5b-9367-4d9c-95dc-0b00babac51a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:01:27.583019 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:27.583000    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bf11f5b-9367-4d9c-95dc-0b00babac51a-kube-api-access-gm9dn" (OuterVolumeSpecName: "kube-api-access-gm9dn") pod "6bf11f5b-9367-4d9c-95dc-0b00babac51a" (UID: "6bf11f5b-9367-4d9c-95dc-0b00babac51a"). InnerVolumeSpecName "kube-api-access-gm9dn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:01:27.583105 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:27.583017    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7be0d75f-3265-4012-8deb-4505b5332f1e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7be0d75f-3265-4012-8deb-4505b5332f1e" (UID: "7be0d75f-3265-4012-8deb-4505b5332f1e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:01:27.583419 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:27.583397    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7be0d75f-3265-4012-8deb-4505b5332f1e-kube-api-access-46vl4" (OuterVolumeSpecName: "kube-api-access-46vl4") pod "7be0d75f-3265-4012-8deb-4505b5332f1e" (UID: "7be0d75f-3265-4012-8deb-4505b5332f1e"). InnerVolumeSpecName "kube-api-access-46vl4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:01:27.681903 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:27.681863    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gm9dn\" (UniqueName: \"kubernetes.io/projected/6bf11f5b-9367-4d9c-95dc-0b00babac51a-kube-api-access-gm9dn\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:01:27.681903 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:27.681895    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7be0d75f-3265-4012-8deb-4505b5332f1e-maas-api-tls\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:01:27.681903 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:27.681905    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46vl4\" (UniqueName: \"kubernetes.io/projected/7be0d75f-3265-4012-8deb-4505b5332f1e-kube-api-access-46vl4\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:01:27.681903 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:27.681914    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7be0d75f-3265-4012-8deb-4505b5332f1e-openshift-service-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:01:27.682209 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:27.681922    2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bf11f5b-9367-4d9c-95dc-0b00babac51a-plugins-config-volume\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:01:27.782734 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:27.782648    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:01:28.163677 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:28.163585    2554 generic.go:356] "Generic (PLEG): container finished" podID="7be0d75f-3265-4012-8deb-4505b5332f1e" containerID="ab155de717eb57f1fb27ad30970c0f34d6a2deae9d7b89243bc0d9841d57ff34" exitCode=0
Jul 30 20:01:28.163677 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:28.163636    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6"
Jul 30 20:01:28.163677 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:28.163666    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6" event={"ID":"7be0d75f-3265-4012-8deb-4505b5332f1e","Type":"ContainerDied","Data":"ab155de717eb57f1fb27ad30970c0f34d6a2deae9d7b89243bc0d9841d57ff34"}
Jul 30 20:01:28.163994 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:28.163726    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6" event={"ID":"7be0d75f-3265-4012-8deb-4505b5332f1e","Type":"ContainerDied","Data":"5bca39ff49dbc0c6efb4df8a4eba0f02e3e757e4f87952dc965633b2573524c8"}
Jul 30 20:01:28.163994 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:28.163754    2554 scope.go:117] "RemoveContainer" containerID="ab155de717eb57f1fb27ad30970c0f34d6a2deae9d7b89243bc0d9841d57ff34"
Jul 30 20:01:28.165073 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:28.165052    2554 generic.go:356] "Generic (PLEG): container finished" podID="6bf11f5b-9367-4d9c-95dc-0b00babac51a" containerID="1d5020a7f43f6aa42a1d207b81d8c8fe51253ab12b85269c8fe64c6ae70b3bef" exitCode=0
Jul 30 20:01:28.165202 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:28.165113    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j"
Jul 30 20:01:28.165202 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:28.165149    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j" event={"ID":"6bf11f5b-9367-4d9c-95dc-0b00babac51a","Type":"ContainerDied","Data":"1d5020a7f43f6aa42a1d207b81d8c8fe51253ab12b85269c8fe64c6ae70b3bef"}
Jul 30 20:01:28.165202 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:28.165195    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j" event={"ID":"6bf11f5b-9367-4d9c-95dc-0b00babac51a","Type":"ContainerDied","Data":"5e781171da34d66e61930b1d66d454c79859fd2e21ec28152bd6346ea25493fd"}
Jul 30 20:01:28.174176 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:28.174156    2554 scope.go:117] "RemoveContainer" containerID="ab155de717eb57f1fb27ad30970c0f34d6a2deae9d7b89243bc0d9841d57ff34"
Jul 30 20:01:28.174462 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:01:28.174439    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab155de717eb57f1fb27ad30970c0f34d6a2deae9d7b89243bc0d9841d57ff34\": container with ID starting with ab155de717eb57f1fb27ad30970c0f34d6a2deae9d7b89243bc0d9841d57ff34 not found: ID does not exist" containerID="ab155de717eb57f1fb27ad30970c0f34d6a2deae9d7b89243bc0d9841d57ff34"
Jul 30 20:01:28.174519 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:28.174475    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab155de717eb57f1fb27ad30970c0f34d6a2deae9d7b89243bc0d9841d57ff34"} err="failed to get container status \"ab155de717eb57f1fb27ad30970c0f34d6a2deae9d7b89243bc0d9841d57ff34\": rpc error: code = NotFound desc = could not find container \"ab155de717eb57f1fb27ad30970c0f34d6a2deae9d7b89243bc0d9841d57ff34\": container with ID starting with ab155de717eb57f1fb27ad30970c0f34d6a2deae9d7b89243bc0d9841d57ff34 not found: ID does not exist"
Jul 30 20:01:28.174519 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:28.174501    2554 scope.go:117] "RemoveContainer" containerID="1d5020a7f43f6aa42a1d207b81d8c8fe51253ab12b85269c8fe64c6ae70b3bef"
Jul 30 20:01:28.182803 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:28.182787    2554 scope.go:117] "RemoveContainer" containerID="1d5020a7f43f6aa42a1d207b81d8c8fe51253ab12b85269c8fe64c6ae70b3bef"
Jul 30 20:01:28.183521 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:01:28.183505    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d5020a7f43f6aa42a1d207b81d8c8fe51253ab12b85269c8fe64c6ae70b3bef\": container with ID starting with 1d5020a7f43f6aa42a1d207b81d8c8fe51253ab12b85269c8fe64c6ae70b3bef not found: ID does not exist" containerID="1d5020a7f43f6aa42a1d207b81d8c8fe51253ab12b85269c8fe64c6ae70b3bef"
Jul 30 20:01:28.183571 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:28.183526    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d5020a7f43f6aa42a1d207b81d8c8fe51253ab12b85269c8fe64c6ae70b3bef"} err="failed to get container status \"1d5020a7f43f6aa42a1d207b81d8c8fe51253ab12b85269c8fe64c6ae70b3bef\": rpc error: code = NotFound desc = could not find container \"1d5020a7f43f6aa42a1d207b81d8c8fe51253ab12b85269c8fe64c6ae70b3bef\": container with ID starting with 1d5020a7f43f6aa42a1d207b81d8c8fe51253ab12b85269c8fe64c6ae70b3bef not found: ID does not exist"
Jul 30 20:01:28.189615 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:28.189591    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6"]
Jul 30 20:01:28.193009 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:28.192990    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-361f6a62-74f4f56d69-4pvk6"]
Jul 30 20:01:28.201742 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:28.201712    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j"]
Jul 30 20:01:28.205429 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:28.205409    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-361f6a62-84dcd69766-mtv7j"]
Jul 30 20:01:28.487180 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:28.487148    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bf11f5b-9367-4d9c-95dc-0b00babac51a" path="/var/lib/kubelet/pods/6bf11f5b-9367-4d9c-95dc-0b00babac51a/volumes"
Jul 30 20:01:28.487560 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:28.487521    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7be0d75f-3265-4012-8deb-4505b5332f1e" path="/var/lib/kubelet/pods/7be0d75f-3265-4012-8deb-4505b5332f1e/volumes"
Jul 30 20:01:33.480914 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:33.480875    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:01:33.534630 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:33.534596    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:01:33.762941 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:33.762858    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:01:33.984938 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:33.984904    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:01:34.112389 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:34.112306    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:01:34.206686 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:34.206657    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:01:34.325739 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:34.325696    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:01:40.052199 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:40.052159    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:01:40.158671 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:40.158636    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:01:40.252782 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:40.252733    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:01:40.415340 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:40.415229    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:01:46.044126 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:46.044082    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:01:47.171073 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:01:47.171034    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:02:14.321207 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:02:14.321109    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:02:14.423494 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:02:14.423461    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:02:14.514157 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:02:14.514124    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjn86"]
Jul 30 20:02:14.514379 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:02:14.514352    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjn86" podUID="9ec1b2c8-32ad-4372-ac57-06dec9fb6077" containerName="manager" containerID="cri-o://e45e3e5a7738ecdfce2cb55420a586b9961148b09075175cb95ed8de9d64f025" gracePeriod=10
Jul 30 20:02:14.865224 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:02:14.865194    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjn86"
Jul 30 20:02:14.937889 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:02:14.937856    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9ec1b2c8-32ad-4372-ac57-06dec9fb6077-extensions-socket-volume\") pod \"9ec1b2c8-32ad-4372-ac57-06dec9fb6077\" (UID: \"9ec1b2c8-32ad-4372-ac57-06dec9fb6077\") "
Jul 30 20:02:14.938072 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:02:14.937901    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkbsn\" (UniqueName: \"kubernetes.io/projected/9ec1b2c8-32ad-4372-ac57-06dec9fb6077-kube-api-access-tkbsn\") pod \"9ec1b2c8-32ad-4372-ac57-06dec9fb6077\" (UID: \"9ec1b2c8-32ad-4372-ac57-06dec9fb6077\") "
Jul 30 20:02:14.938272 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:02:14.938241    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ec1b2c8-32ad-4372-ac57-06dec9fb6077-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9ec1b2c8-32ad-4372-ac57-06dec9fb6077" (UID: "9ec1b2c8-32ad-4372-ac57-06dec9fb6077"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:02:14.939930 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:02:14.939906    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ec1b2c8-32ad-4372-ac57-06dec9fb6077-kube-api-access-tkbsn" (OuterVolumeSpecName: "kube-api-access-tkbsn") pod "9ec1b2c8-32ad-4372-ac57-06dec9fb6077" (UID: "9ec1b2c8-32ad-4372-ac57-06dec9fb6077"). InnerVolumeSpecName "kube-api-access-tkbsn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:02:15.038925 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:02:15.038838    2554 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9ec1b2c8-32ad-4372-ac57-06dec9fb6077-extensions-socket-volume\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:02:15.038925 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:02:15.038871    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkbsn\" (UniqueName: \"kubernetes.io/projected/9ec1b2c8-32ad-4372-ac57-06dec9fb6077-kube-api-access-tkbsn\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:02:15.442575 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:02:15.442480    2554 generic.go:356] "Generic (PLEG): container finished" podID="9ec1b2c8-32ad-4372-ac57-06dec9fb6077" containerID="e45e3e5a7738ecdfce2cb55420a586b9961148b09075175cb95ed8de9d64f025" exitCode=0
Jul 30 20:02:15.442575 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:02:15.442536    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjn86" event={"ID":"9ec1b2c8-32ad-4372-ac57-06dec9fb6077","Type":"ContainerDied","Data":"e45e3e5a7738ecdfce2cb55420a586b9961148b09075175cb95ed8de9d64f025"}
Jul 30 20:02:15.442575 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:02:15.442538    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjn86"
Jul 30 20:02:15.443104 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:02:15.442604    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjn86" event={"ID":"9ec1b2c8-32ad-4372-ac57-06dec9fb6077","Type":"ContainerDied","Data":"c11abc6b5b4f1d27a9393193fdcc4182e81f6cb611cbe47422af4b542d5123e3"}
Jul 30 20:02:15.443104 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:02:15.442627    2554 scope.go:117] "RemoveContainer" containerID="e45e3e5a7738ecdfce2cb55420a586b9961148b09075175cb95ed8de9d64f025"
Jul 30 20:02:15.453786 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:02:15.453758    2554 scope.go:117] "RemoveContainer" containerID="e45e3e5a7738ecdfce2cb55420a586b9961148b09075175cb95ed8de9d64f025"
Jul 30 20:02:15.454072 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:02:15.454051    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e45e3e5a7738ecdfce2cb55420a586b9961148b09075175cb95ed8de9d64f025\": container with ID starting with e45e3e5a7738ecdfce2cb55420a586b9961148b09075175cb95ed8de9d64f025 not found: ID does not exist" containerID="e45e3e5a7738ecdfce2cb55420a586b9961148b09075175cb95ed8de9d64f025"
Jul 30 20:02:15.454183 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:02:15.454079    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e45e3e5a7738ecdfce2cb55420a586b9961148b09075175cb95ed8de9d64f025"} err="failed to get container status \"e45e3e5a7738ecdfce2cb55420a586b9961148b09075175cb95ed8de9d64f025\": rpc error: code = NotFound desc = could not find container \"e45e3e5a7738ecdfce2cb55420a586b9961148b09075175cb95ed8de9d64f025\": container with ID starting with e45e3e5a7738ecdfce2cb55420a586b9961148b09075175cb95ed8de9d64f025 not found: ID does not exist"
Jul 30 20:02:15.465427 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:02:15.465401    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjn86"]
Jul 30 20:02:15.470496 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:02:15.470476    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bjn86"]
Jul 30 20:02:16.486764 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:02:16.486732    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ec1b2c8-32ad-4372-ac57-06dec9fb6077" path="/var/lib/kubelet/pods/9ec1b2c8-32ad-4372-ac57-06dec9fb6077/volumes"
Jul 30 20:03:30.133082 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:30.132978    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w5ffc"]
Jul 30 20:03:30.135430 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:30.133440    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ec1b2c8-32ad-4372-ac57-06dec9fb6077" containerName="manager"
Jul 30 20:03:30.135430 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:30.133456    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ec1b2c8-32ad-4372-ac57-06dec9fb6077" containerName="manager"
Jul 30 20:03:30.135430 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:30.133464    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7be0d75f-3265-4012-8deb-4505b5332f1e" containerName="maas-api"
Jul 30 20:03:30.135430 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:30.133470    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="7be0d75f-3265-4012-8deb-4505b5332f1e" containerName="maas-api"
Jul 30 20:03:30.135430 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:30.133485    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bf11f5b-9367-4d9c-95dc-0b00babac51a" containerName="payload-processing"
Jul 30 20:03:30.135430 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:30.133490    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bf11f5b-9367-4d9c-95dc-0b00babac51a" containerName="payload-processing"
Jul 30 20:03:30.135430 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:30.133557    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bf11f5b-9367-4d9c-95dc-0b00babac51a" containerName="payload-processing"
Jul 30 20:03:30.135430 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:30.133569    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="7be0d75f-3265-4012-8deb-4505b5332f1e" containerName="maas-api"
Jul 30 20:03:30.135430 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:30.133577    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ec1b2c8-32ad-4372-ac57-06dec9fb6077" containerName="manager"
Jul 30 20:03:30.136618 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:30.136595    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w5ffc"
Jul 30 20:03:30.139286 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:30.139266    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mpd9r\""
Jul 30 20:03:30.148557 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:30.148528    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w5ffc"]
Jul 30 20:03:30.256302 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:30.256261    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cv67\" (UniqueName: \"kubernetes.io/projected/e7accbac-7008-46b2-b1c1-37a50151e649-kube-api-access-2cv67\") pod \"kuadrant-operator-controller-manager-86878c769f-w5ffc\" (UID: \"e7accbac-7008-46b2-b1c1-37a50151e649\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w5ffc"
Jul 30 20:03:30.256302 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:30.256306    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7accbac-7008-46b2-b1c1-37a50151e649-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-w5ffc\" (UID: \"e7accbac-7008-46b2-b1c1-37a50151e649\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w5ffc"
Jul 30 20:03:30.357524 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:30.357480    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cv67\" (UniqueName: \"kubernetes.io/projected/e7accbac-7008-46b2-b1c1-37a50151e649-kube-api-access-2cv67\") pod \"kuadrant-operator-controller-manager-86878c769f-w5ffc\" (UID: \"e7accbac-7008-46b2-b1c1-37a50151e649\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w5ffc"
Jul 30 20:03:30.357524 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:30.357532    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7accbac-7008-46b2-b1c1-37a50151e649-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-w5ffc\" (UID: \"e7accbac-7008-46b2-b1c1-37a50151e649\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w5ffc"
Jul 30 20:03:30.357942 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:30.357918    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7accbac-7008-46b2-b1c1-37a50151e649-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-w5ffc\" (UID: \"e7accbac-7008-46b2-b1c1-37a50151e649\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w5ffc"
Jul 30 20:03:30.365919 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:30.365891    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cv67\" (UniqueName: \"kubernetes.io/projected/e7accbac-7008-46b2-b1c1-37a50151e649-kube-api-access-2cv67\") pod \"kuadrant-operator-controller-manager-86878c769f-w5ffc\" (UID: \"e7accbac-7008-46b2-b1c1-37a50151e649\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w5ffc"
Jul 30 20:03:30.448072 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:30.447940    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w5ffc"
Jul 30 20:03:30.609668 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:30.609638    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w5ffc"]
Jul 30 20:03:30.611436 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:03:30.611408    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7accbac_7008_46b2_b1c1_37a50151e649.slice/crio-fca32e781f477a81180be177c2d2d756e159abc50c139a0389541838488fd35a WatchSource:0}: Error finding container fca32e781f477a81180be177c2d2d756e159abc50c139a0389541838488fd35a: Status 404 returned error can't find the container with id fca32e781f477a81180be177c2d2d756e159abc50c139a0389541838488fd35a
Jul 30 20:03:30.870706 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:30.870661    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w5ffc" event={"ID":"e7accbac-7008-46b2-b1c1-37a50151e649","Type":"ContainerStarted","Data":"6f7c2ed348535e49b03a44cab8c0681d8eff339642c46b1f756d5e3548df9607"}
Jul 30 20:03:30.870890 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:30.870733    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w5ffc" event={"ID":"e7accbac-7008-46b2-b1c1-37a50151e649","Type":"ContainerStarted","Data":"fca32e781f477a81180be177c2d2d756e159abc50c139a0389541838488fd35a"}
Jul 30 20:03:30.870890 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:30.870766    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w5ffc"
Jul 30 20:03:30.897501 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:30.897435    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w5ffc" podStartSLOduration=0.897417858 podStartE2EDuration="897.417858ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:03:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:03:30.895679388 +0000 UTC m=+2338.860260506" watchObservedRunningTime="2026-07-30 20:03:30.897417858 +0000 UTC m=+2338.861998938"
Jul 30 20:03:32.790288 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:32.790260    2554 scope.go:117] "RemoveContainer" containerID="21db08122ec3ddf95940e122d9ccfb6366701c2505717e18801735b18995734d"
Jul 30 20:03:32.799132 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:32.799111    2554 scope.go:117] "RemoveContainer" containerID="6bfc887a343607fedb1d9cdf88816e0f6b20459c8dbe7c2c5c969cdd9ab899af"
Jul 30 20:03:41.878674 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:41.878642    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w5ffc"
Jul 30 20:03:48.405392 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:48.405357    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:03:48.710205 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:48.710170    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:03:48.972831 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:48.972731    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:03:51.417669 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:51.417603    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:03:51.505432 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:51.505399    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:03:52.473438 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:52.473402    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r"]
Jul 30 20:03:52.477841 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:52.477820    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r"
Jul 30 20:03:52.480342 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:52.480321    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-f6ff274e-serving-cert\""
Jul 30 20:03:52.489135 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:52.489108    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r"]
Jul 30 20:03:52.507820 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:52.507789    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:03:52.573224 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:52.573196    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00694c4f-a5f2-413d-b8d6-bd01beb500c0-openshift-service-ca\") pod \"maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r\" (UID: \"00694c4f-a5f2-413d-b8d6-bd01beb500c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r"
Jul 30 20:03:52.573402 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:52.573269    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00694c4f-a5f2-413d-b8d6-bd01beb500c0-maas-api-tls\") pod \"maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r\" (UID: \"00694c4f-a5f2-413d-b8d6-bd01beb500c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r"
Jul 30 20:03:52.573402 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:52.573347    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thgw4\" (UniqueName: \"kubernetes.io/projected/00694c4f-a5f2-413d-b8d6-bd01beb500c0-kube-api-access-thgw4\") pod \"maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r\" (UID: \"00694c4f-a5f2-413d-b8d6-bd01beb500c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r"
Jul 30 20:03:52.674374 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:52.674331    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00694c4f-a5f2-413d-b8d6-bd01beb500c0-maas-api-tls\") pod \"maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r\" (UID: \"00694c4f-a5f2-413d-b8d6-bd01beb500c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r"
Jul 30 20:03:52.674592 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:52.674392    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thgw4\" (UniqueName: \"kubernetes.io/projected/00694c4f-a5f2-413d-b8d6-bd01beb500c0-kube-api-access-thgw4\") pod \"maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r\" (UID: \"00694c4f-a5f2-413d-b8d6-bd01beb500c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r"
Jul 30 20:03:52.674592 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:52.674443    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00694c4f-a5f2-413d-b8d6-bd01beb500c0-openshift-service-ca\") pod \"maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r\" (UID: \"00694c4f-a5f2-413d-b8d6-bd01beb500c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r"
Jul 30 20:03:52.674592 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:03:52.674516    2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-serving-cert: secret "maas-api-e2e-mt-f6ff274e-serving-cert" not found
Jul 30 20:03:52.674791 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:03:52.674600    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00694c4f-a5f2-413d-b8d6-bd01beb500c0-maas-api-tls podName:00694c4f-a5f2-413d-b8d6-bd01beb500c0 nodeName:}" failed. No retries permitted until 2026-07-30 20:03:53.174578872 +0000 UTC m=+2361.139159931 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/00694c4f-a5f2-413d-b8d6-bd01beb500c0-maas-api-tls") pod "maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r" (UID: "00694c4f-a5f2-413d-b8d6-bd01beb500c0") : secret "maas-api-e2e-mt-f6ff274e-serving-cert" not found
Jul 30 20:03:52.675079 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:52.675062    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00694c4f-a5f2-413d-b8d6-bd01beb500c0-openshift-service-ca\") pod \"maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r\" (UID: \"00694c4f-a5f2-413d-b8d6-bd01beb500c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r"
Jul 30 20:03:52.686108 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:52.686083    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thgw4\" (UniqueName: \"kubernetes.io/projected/00694c4f-a5f2-413d-b8d6-bd01beb500c0-kube-api-access-thgw4\") pod \"maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r\" (UID: \"00694c4f-a5f2-413d-b8d6-bd01beb500c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r"
Jul 30 20:03:52.839658 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:52.839549    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:03:52.993391 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:52.993354    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:03:53.089295 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:53.089263    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:03:53.139130 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:53.139040    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s"]
Jul 30 20:03:53.142555 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:53.142537    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s"
Jul 30 20:03:53.145327 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:53.145307    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-f6ff274e-dockercfg-k7ft9\""
Jul 30 20:03:53.145327 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:53.145320    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-f6ff274e\""
Jul 30 20:03:53.154359 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:53.154328    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s"]
Jul 30 20:03:53.180235 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:53.180202    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00694c4f-a5f2-413d-b8d6-bd01beb500c0-maas-api-tls\") pod \"maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r\" (UID: \"00694c4f-a5f2-413d-b8d6-bd01beb500c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r"
Jul 30 20:03:53.182782 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:53.182749    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00694c4f-a5f2-413d-b8d6-bd01beb500c0-maas-api-tls\") pod \"maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r\" (UID: \"00694c4f-a5f2-413d-b8d6-bd01beb500c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r"
Jul 30 20:03:53.281441 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:53.281405    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca6f8635-03b6-424e-aaa9-efec817a7bdc-plugins-config-volume\") pod \"payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s\" (UID: \"ca6f8635-03b6-424e-aaa9-efec817a7bdc\") " pod="openshift-ingress/payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s"
Jul 30 20:03:53.281614 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:53.281466    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rz8mr\" (UniqueName: \"kubernetes.io/projected/ca6f8635-03b6-424e-aaa9-efec817a7bdc-kube-api-access-rz8mr\") pod \"payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s\" (UID: \"ca6f8635-03b6-424e-aaa9-efec817a7bdc\") " pod="openshift-ingress/payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s"
Jul 30 20:03:53.382415 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:53.382372    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca6f8635-03b6-424e-aaa9-efec817a7bdc-plugins-config-volume\") pod \"payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s\" (UID: \"ca6f8635-03b6-424e-aaa9-efec817a7bdc\") " pod="openshift-ingress/payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s"
Jul 30 20:03:53.382614 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:53.382436    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rz8mr\" (UniqueName: \"kubernetes.io/projected/ca6f8635-03b6-424e-aaa9-efec817a7bdc-kube-api-access-rz8mr\") pod \"payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s\" (UID: \"ca6f8635-03b6-424e-aaa9-efec817a7bdc\") " pod="openshift-ingress/payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s"
Jul 30 20:03:53.383199 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:53.383178    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca6f8635-03b6-424e-aaa9-efec817a7bdc-plugins-config-volume\") pod \"payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s\" (UID: \"ca6f8635-03b6-424e-aaa9-efec817a7bdc\") " pod="openshift-ingress/payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s"
Jul 30 20:03:53.390206 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:53.390136    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rz8mr\" (UniqueName: \"kubernetes.io/projected/ca6f8635-03b6-424e-aaa9-efec817a7bdc-kube-api-access-rz8mr\") pod \"payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s\" (UID: \"ca6f8635-03b6-424e-aaa9-efec817a7bdc\") " pod="openshift-ingress/payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s"
Jul 30 20:03:53.390775 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:53.390749    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r"
Jul 30 20:03:53.455045 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:53.455009    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s"
Jul 30 20:03:53.610193 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:53.610170    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s"]
Jul 30 20:03:53.612583 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:03:53.612549    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca6f8635_03b6_424e_aaa9_efec817a7bdc.slice/crio-e94b57fd581e412e639f096655f92ee8f8f250294d3affacb0b6bfa1145b412a WatchSource:0}: Error finding container e94b57fd581e412e639f096655f92ee8f8f250294d3affacb0b6bfa1145b412a: Status 404 returned error can't find the container with id e94b57fd581e412e639f096655f92ee8f8f250294d3affacb0b6bfa1145b412a
Jul 30 20:03:53.748573 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:53.748535    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r"]
Jul 30 20:03:53.750049 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:03:53.750021    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00694c4f_a5f2_413d_b8d6_bd01beb500c0.slice/crio-c20c3f7ee0a876840c2831866fe2f91e1c18cc248ac674a82eca05cdf3c90161 WatchSource:0}: Error finding container c20c3f7ee0a876840c2831866fe2f91e1c18cc248ac674a82eca05cdf3c90161: Status 404 returned error can't find the container with id c20c3f7ee0a876840c2831866fe2f91e1c18cc248ac674a82eca05cdf3c90161
Jul 30 20:03:53.751563 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:53.751547    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 20:03:54.003135 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:54.003096    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r" event={"ID":"00694c4f-a5f2-413d-b8d6-bd01beb500c0","Type":"ContainerStarted","Data":"c20c3f7ee0a876840c2831866fe2f91e1c18cc248ac674a82eca05cdf3c90161"}
Jul 30 20:03:54.004638 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:54.004607    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s" event={"ID":"ca6f8635-03b6-424e-aaa9-efec817a7bdc","Type":"ContainerStarted","Data":"dd7dc2f02d93f0236c13d994054f4ef121f3161f948e0f4ec4adafcf355de15b"}
Jul 30 20:03:54.004768 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:54.004652    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s" event={"ID":"ca6f8635-03b6-424e-aaa9-efec817a7bdc","Type":"ContainerStarted","Data":"e94b57fd581e412e639f096655f92ee8f8f250294d3affacb0b6bfa1145b412a"}
Jul 30 20:03:54.004768 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:54.004702    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s"
Jul 30 20:03:54.026318 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:54.026250    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s" podStartSLOduration=1.026229405 podStartE2EDuration="1.026229405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:03:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:03:54.022117082 +0000 UTC m=+2361.986698163" watchObservedRunningTime="2026-07-30 20:03:54.026229405 +0000 UTC m=+2361.990810490"
Jul 30 20:03:55.012832 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:55.012789    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r" event={"ID":"00694c4f-a5f2-413d-b8d6-bd01beb500c0","Type":"ContainerStarted","Data":"f25e75d4c6880761d04cad97060c2aaae86f1b594672d10d9b397b5d2ad722fd"}
Jul 30 20:03:55.013367 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:55.013053    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r"
Jul 30 20:03:55.031419 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:03:55.031368    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r" podStartSLOduration=2.4579337470000002 podStartE2EDuration="3.031351808s" totalImagesPullingTime="573.418061ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:03:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:03:55.028287776 +0000 UTC m=+2362.992868870" watchObservedRunningTime="2026-07-30 20:03:55.031351808 +0000 UTC m=+2362.995932889"
Jul 30 20:04:01.025229 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:04:01.025198    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r"
Jul 30 20:04:05.015078 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:04:05.015047    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s"
Jul 30 20:04:16.354745 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:04:16.354707    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:04:21.373259 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:04:21.373219    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:04:32.706377 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:04:32.706346    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/1.log"
Jul 30 20:04:32.711179 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:04:32.711155    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 20:04:32.714963 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:04:32.714934    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/1.log"
Jul 30 20:04:32.719166 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:04:32.719149    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 20:04:32.823398 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:04:32.823365    2554 scope.go:117] "RemoveContainer" containerID="e60ae343ce5cbc8deeda46d6be08e329d75931ec9cee1f727a5ffb70a053d681"
Jul 30 20:04:32.832073 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:04:32.832055    2554 scope.go:117] "RemoveContainer" containerID="29cf6ae5095e65ad3ccfe6f7b1d34bfb59a745c7822864fd734de2d1cde56d0e"
Jul 30 20:05:03.382064 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:03.381974    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:05:03.483462 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:03.483427    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:05:11.405538 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:11.405496    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r"]
Jul 30 20:05:11.406044 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:11.405766    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r" podUID="00694c4f-a5f2-413d-b8d6-bd01beb500c0" containerName="maas-api" containerID="cri-o://f25e75d4c6880761d04cad97060c2aaae86f1b594672d10d9b397b5d2ad722fd" gracePeriod=30
Jul 30 20:05:11.430490 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:11.430461    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:05:11.462566 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:11.462539    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s"]
Jul 30 20:05:11.462769 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:11.462742    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s" podUID="ca6f8635-03b6-424e-aaa9-efec817a7bdc" containerName="payload-processing" containerID="cri-o://dd7dc2f02d93f0236c13d994054f4ef121f3161f948e0f4ec4adafcf355de15b" gracePeriod=30
Jul 30 20:05:11.491654 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:11.491619    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:05:11.547739 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:11.547698    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:05:11.604292 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:11.604197    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:05:11.731437 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:11.731405    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r"
Jul 30 20:05:11.757266 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:11.757236    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s"
Jul 30 20:05:11.839946 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:11.839910    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00694c4f-a5f2-413d-b8d6-bd01beb500c0-maas-api-tls\") pod \"00694c4f-a5f2-413d-b8d6-bd01beb500c0\" (UID: \"00694c4f-a5f2-413d-b8d6-bd01beb500c0\") "
Jul 30 20:05:11.840156 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:11.840000    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00694c4f-a5f2-413d-b8d6-bd01beb500c0-openshift-service-ca\") pod \"00694c4f-a5f2-413d-b8d6-bd01beb500c0\" (UID: \"00694c4f-a5f2-413d-b8d6-bd01beb500c0\") "
Jul 30 20:05:11.840156 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:11.840033    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thgw4\" (UniqueName: \"kubernetes.io/projected/00694c4f-a5f2-413d-b8d6-bd01beb500c0-kube-api-access-thgw4\") pod \"00694c4f-a5f2-413d-b8d6-bd01beb500c0\" (UID: \"00694c4f-a5f2-413d-b8d6-bd01beb500c0\") "
Jul 30 20:05:11.840156 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:11.840052    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rz8mr\" (UniqueName: \"kubernetes.io/projected/ca6f8635-03b6-424e-aaa9-efec817a7bdc-kube-api-access-rz8mr\") pod \"ca6f8635-03b6-424e-aaa9-efec817a7bdc\" (UID: \"ca6f8635-03b6-424e-aaa9-efec817a7bdc\") "
Jul 30 20:05:11.840156 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:11.840105    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca6f8635-03b6-424e-aaa9-efec817a7bdc-plugins-config-volume\") pod \"ca6f8635-03b6-424e-aaa9-efec817a7bdc\" (UID: \"ca6f8635-03b6-424e-aaa9-efec817a7bdc\") "
Jul 30 20:05:11.840483 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:11.840452    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00694c4f-a5f2-413d-b8d6-bd01beb500c0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "00694c4f-a5f2-413d-b8d6-bd01beb500c0" (UID: "00694c4f-a5f2-413d-b8d6-bd01beb500c0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:05:11.840618 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:11.840569    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca6f8635-03b6-424e-aaa9-efec817a7bdc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ca6f8635-03b6-424e-aaa9-efec817a7bdc" (UID: "ca6f8635-03b6-424e-aaa9-efec817a7bdc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:05:11.842202 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:11.842176    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00694c4f-a5f2-413d-b8d6-bd01beb500c0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "00694c4f-a5f2-413d-b8d6-bd01beb500c0" (UID: "00694c4f-a5f2-413d-b8d6-bd01beb500c0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:05:11.842296 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:11.842235    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00694c4f-a5f2-413d-b8d6-bd01beb500c0-kube-api-access-thgw4" (OuterVolumeSpecName: "kube-api-access-thgw4") pod "00694c4f-a5f2-413d-b8d6-bd01beb500c0" (UID: "00694c4f-a5f2-413d-b8d6-bd01beb500c0"). InnerVolumeSpecName "kube-api-access-thgw4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:05:11.842296 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:11.842257    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca6f8635-03b6-424e-aaa9-efec817a7bdc-kube-api-access-rz8mr" (OuterVolumeSpecName: "kube-api-access-rz8mr") pod "ca6f8635-03b6-424e-aaa9-efec817a7bdc" (UID: "ca6f8635-03b6-424e-aaa9-efec817a7bdc"). InnerVolumeSpecName "kube-api-access-rz8mr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:05:11.940935 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:11.940847    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thgw4\" (UniqueName: \"kubernetes.io/projected/00694c4f-a5f2-413d-b8d6-bd01beb500c0-kube-api-access-thgw4\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:05:11.940935 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:11.940876    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rz8mr\" (UniqueName: \"kubernetes.io/projected/ca6f8635-03b6-424e-aaa9-efec817a7bdc-kube-api-access-rz8mr\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:05:11.940935 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:11.940886    2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca6f8635-03b6-424e-aaa9-efec817a7bdc-plugins-config-volume\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:05:11.940935 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:11.940896    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00694c4f-a5f2-413d-b8d6-bd01beb500c0-maas-api-tls\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:05:11.940935 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:11.940904    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00694c4f-a5f2-413d-b8d6-bd01beb500c0-openshift-service-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:05:12.473037 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:12.473003    2554 generic.go:356] "Generic (PLEG): container finished" podID="ca6f8635-03b6-424e-aaa9-efec817a7bdc" containerID="dd7dc2f02d93f0236c13d994054f4ef121f3161f948e0f4ec4adafcf355de15b" exitCode=0
Jul 30 20:05:12.473453 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:12.473048    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s"
Jul 30 20:05:12.473453 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:12.473085    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s" event={"ID":"ca6f8635-03b6-424e-aaa9-efec817a7bdc","Type":"ContainerDied","Data":"dd7dc2f02d93f0236c13d994054f4ef121f3161f948e0f4ec4adafcf355de15b"}
Jul 30 20:05:12.473453 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:12.473143    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s" event={"ID":"ca6f8635-03b6-424e-aaa9-efec817a7bdc","Type":"ContainerDied","Data":"e94b57fd581e412e639f096655f92ee8f8f250294d3affacb0b6bfa1145b412a"}
Jul 30 20:05:12.473453 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:12.473165    2554 scope.go:117] "RemoveContainer" containerID="dd7dc2f02d93f0236c13d994054f4ef121f3161f948e0f4ec4adafcf355de15b"
Jul 30 20:05:12.474724 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:12.474700    2554 generic.go:356] "Generic (PLEG): container finished" podID="00694c4f-a5f2-413d-b8d6-bd01beb500c0" containerID="f25e75d4c6880761d04cad97060c2aaae86f1b594672d10d9b397b5d2ad722fd" exitCode=0
Jul 30 20:05:12.474836 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:12.474737    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r"
Jul 30 20:05:12.474836 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:12.474753    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r" event={"ID":"00694c4f-a5f2-413d-b8d6-bd01beb500c0","Type":"ContainerDied","Data":"f25e75d4c6880761d04cad97060c2aaae86f1b594672d10d9b397b5d2ad722fd"}
Jul 30 20:05:12.474836 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:12.474783    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r" event={"ID":"00694c4f-a5f2-413d-b8d6-bd01beb500c0","Type":"ContainerDied","Data":"c20c3f7ee0a876840c2831866fe2f91e1c18cc248ac674a82eca05cdf3c90161"}
Jul 30 20:05:12.484218 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:12.484199    2554 scope.go:117] "RemoveContainer" containerID="dd7dc2f02d93f0236c13d994054f4ef121f3161f948e0f4ec4adafcf355de15b"
Jul 30 20:05:12.484479 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:05:12.484455    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd7dc2f02d93f0236c13d994054f4ef121f3161f948e0f4ec4adafcf355de15b\": container with ID starting with dd7dc2f02d93f0236c13d994054f4ef121f3161f948e0f4ec4adafcf355de15b not found: ID does not exist" containerID="dd7dc2f02d93f0236c13d994054f4ef121f3161f948e0f4ec4adafcf355de15b"
Jul 30 20:05:12.484541 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:12.484489    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd7dc2f02d93f0236c13d994054f4ef121f3161f948e0f4ec4adafcf355de15b"} err="failed to get container status \"dd7dc2f02d93f0236c13d994054f4ef121f3161f948e0f4ec4adafcf355de15b\": rpc error: code = NotFound desc = could not find container \"dd7dc2f02d93f0236c13d994054f4ef121f3161f948e0f4ec4adafcf355de15b\": container with ID starting with dd7dc2f02d93f0236c13d994054f4ef121f3161f948e0f4ec4adafcf355de15b not found: ID does not exist"
Jul 30 20:05:12.484541 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:12.484508    2554 scope.go:117] "RemoveContainer" containerID="f25e75d4c6880761d04cad97060c2aaae86f1b594672d10d9b397b5d2ad722fd"
Jul 30 20:05:12.493393 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:12.493377    2554 scope.go:117] "RemoveContainer" containerID="f25e75d4c6880761d04cad97060c2aaae86f1b594672d10d9b397b5d2ad722fd"
Jul 30 20:05:12.493623 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:05:12.493607    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f25e75d4c6880761d04cad97060c2aaae86f1b594672d10d9b397b5d2ad722fd\": container with ID starting with f25e75d4c6880761d04cad97060c2aaae86f1b594672d10d9b397b5d2ad722fd not found: ID does not exist" containerID="f25e75d4c6880761d04cad97060c2aaae86f1b594672d10d9b397b5d2ad722fd"
Jul 30 20:05:12.493665 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:12.493630    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f25e75d4c6880761d04cad97060c2aaae86f1b594672d10d9b397b5d2ad722fd"} err="failed to get container status \"f25e75d4c6880761d04cad97060c2aaae86f1b594672d10d9b397b5d2ad722fd\": rpc error: code = NotFound desc = could not find container \"f25e75d4c6880761d04cad97060c2aaae86f1b594672d10d9b397b5d2ad722fd\": container with ID starting with f25e75d4c6880761d04cad97060c2aaae86f1b594672d10d9b397b5d2ad722fd not found: ID does not exist"
Jul 30 20:05:12.503195 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:12.503175    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r"]
Jul 30 20:05:12.507232 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:12.507210    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f6ff274e-796c85c686-xjk9r"]
Jul 30 20:05:12.517229 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:12.517204    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s"]
Jul 30 20:05:12.522881 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:12.522852    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f6ff274e-df787b5c8-xhh5s"]
Jul 30 20:05:14.487638 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:14.487603    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00694c4f-a5f2-413d-b8d6-bd01beb500c0" path="/var/lib/kubelet/pods/00694c4f-a5f2-413d-b8d6-bd01beb500c0/volumes"
Jul 30 20:05:14.488053 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:14.488010    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca6f8635-03b6-424e-aaa9-efec817a7bdc" path="/var/lib/kubelet/pods/ca6f8635-03b6-424e-aaa9-efec817a7bdc/volumes"
Jul 30 20:05:14.910636 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:14.910536    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:05:19.989904 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:19.989861    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:05:50.721249 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:50.721211    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:05:51.069041 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:51.068933    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:05:56.601442 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:56.601403    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:05:56.661425 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:56.661390    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:05:56.885931 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:56.885837    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:05:57.114340 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:05:57.114296    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:06:18.640200 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:06:18.640161    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:06:18.737287 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:06:18.737250    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:06:18.837333 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:06:18.837288    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:06:19.525720 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:06:19.525683    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:06:19.612241 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:06:19.612208    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:06:19.711927 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:06:19.711885    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:06:21.015150 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:06:21.015114    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:06:27.028655 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:06:27.028563    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:06:27.129235 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:06:27.129186    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:06:27.765996 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:06:27.765937    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:06:33.451912 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:06:33.451871    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:06:33.506530 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:06:33.506477    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:06:33.578487 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:06:33.578453    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:06:33.674528 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:06:33.674485    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:06:44.199317 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:06:44.199272    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:06:44.473846 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:06:44.473807    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:07:05.428559 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:05.428508    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:07:11.512605 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:11.512566    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:07:11.608253 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:11.608213    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:07:11.864919 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:11.864823    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:07:12.163163 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:12.163083    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:07:12.288481 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:12.288449    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:07:15.218270 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:15.218229    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:07:15.318594 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:15.318560    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:07:16.271606 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.271567    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8"]
Jul 30 20:07:16.272111 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.272072    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca6f8635-03b6-424e-aaa9-efec817a7bdc" containerName="payload-processing"
Jul 30 20:07:16.272111 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.272087    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca6f8635-03b6-424e-aaa9-efec817a7bdc" containerName="payload-processing"
Jul 30 20:07:16.272231 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.272115    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00694c4f-a5f2-413d-b8d6-bd01beb500c0" containerName="maas-api"
Jul 30 20:07:16.272231 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.272121    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="00694c4f-a5f2-413d-b8d6-bd01beb500c0" containerName="maas-api"
Jul 30 20:07:16.272231 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.272181    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="00694c4f-a5f2-413d-b8d6-bd01beb500c0" containerName="maas-api"
Jul 30 20:07:16.272231 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.272193    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca6f8635-03b6-424e-aaa9-efec817a7bdc" containerName="payload-processing"
Jul 30 20:07:16.275320 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.275301    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8"
Jul 30 20:07:16.277795 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.277768    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-37abda-serving-cert\""
Jul 30 20:07:16.284167 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.283611    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8"]
Jul 30 20:07:16.299506 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.299475    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:07:16.418537 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.418503    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa-maas-api-tls\") pod \"maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8\" (UID: \"01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8"
Jul 30 20:07:16.418722 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.418582    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7x78\" (UniqueName: \"kubernetes.io/projected/01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa-kube-api-access-t7x78\") pod \"maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8\" (UID: \"01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8"
Jul 30 20:07:16.418722 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.418644    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa-openshift-service-ca\") pod \"maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8\" (UID: \"01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8"
Jul 30 20:07:16.519596 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.519568    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa-openshift-service-ca\") pod \"maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8\" (UID: \"01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8"
Jul 30 20:07:16.519780 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.519628    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa-maas-api-tls\") pod \"maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8\" (UID: \"01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8"
Jul 30 20:07:16.519780 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.519669    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7x78\" (UniqueName: \"kubernetes.io/projected/01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa-kube-api-access-t7x78\") pod \"maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8\" (UID: \"01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8"
Jul 30 20:07:16.520249 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.520228    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa-openshift-service-ca\") pod \"maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8\" (UID: \"01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8"
Jul 30 20:07:16.522227 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.522156    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa-maas-api-tls\") pod \"maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8\" (UID: \"01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8"
Jul 30 20:07:16.530652 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.530622    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7x78\" (UniqueName: \"kubernetes.io/projected/01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa-kube-api-access-t7x78\") pod \"maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8\" (UID: \"01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8"
Jul 30 20:07:16.556629 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.556598    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:07:16.588315 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.588287    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8"
Jul 30 20:07:16.733629 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.733591    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8"]
Jul 30 20:07:16.733757 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:07:16.733693    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01a9b07c_fe8c_4d58_aa7a_6a7ecd0af4aa.slice/crio-4a11b5f1b17b9820bd3e7328d637a66949772801e1633628765c8b1b5130e35b WatchSource:0}: Error finding container 4a11b5f1b17b9820bd3e7328d637a66949772801e1633628765c8b1b5130e35b: Status 404 returned error can't find the container with id 4a11b5f1b17b9820bd3e7328d637a66949772801e1633628765c8b1b5130e35b
Jul 30 20:07:16.940705 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.940621    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7"]
Jul 30 20:07:16.945648 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.945627    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7"
Jul 30 20:07:16.948472 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.948446    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-37abda-dockercfg-8qkwp\""
Jul 30 20:07:16.948610 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.948474    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-37abda\""
Jul 30 20:07:16.954700 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.954666    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7"]
Jul 30 20:07:16.960987 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:16.960930    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:07:17.024661 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:17.024612    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq4gz\" (UniqueName: \"kubernetes.io/projected/460c93d5-cd74-436c-8fc4-1d4d7f42348a-kube-api-access-xq4gz\") pod \"payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7\" (UID: \"460c93d5-cd74-436c-8fc4-1d4d7f42348a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7"
Jul 30 20:07:17.024843 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:17.024702    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/460c93d5-cd74-436c-8fc4-1d4d7f42348a-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7\" (UID: \"460c93d5-cd74-436c-8fc4-1d4d7f42348a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7"
Jul 30 20:07:17.062705 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:17.062670    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:07:17.125797 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:17.125766    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq4gz\" (UniqueName: \"kubernetes.io/projected/460c93d5-cd74-436c-8fc4-1d4d7f42348a-kube-api-access-xq4gz\") pod \"payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7\" (UID: \"460c93d5-cd74-436c-8fc4-1d4d7f42348a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7"
Jul 30 20:07:17.125992 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:17.125821    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/460c93d5-cd74-436c-8fc4-1d4d7f42348a-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7\" (UID: \"460c93d5-cd74-436c-8fc4-1d4d7f42348a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7"
Jul 30 20:07:17.126611 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:17.126591    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/460c93d5-cd74-436c-8fc4-1d4d7f42348a-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7\" (UID: \"460c93d5-cd74-436c-8fc4-1d4d7f42348a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7"
Jul 30 20:07:17.133738 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:17.133712    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq4gz\" (UniqueName: \"kubernetes.io/projected/460c93d5-cd74-436c-8fc4-1d4d7f42348a-kube-api-access-xq4gz\") pod \"payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7\" (UID: \"460c93d5-cd74-436c-8fc4-1d4d7f42348a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7"
Jul 30 20:07:17.194843 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:17.194775    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8" event={"ID":"01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa","Type":"ContainerStarted","Data":"4a11b5f1b17b9820bd3e7328d637a66949772801e1633628765c8b1b5130e35b"}
Jul 30 20:07:17.257514 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:17.257480    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7"
Jul 30 20:07:17.391938 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:17.391908    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7"]
Jul 30 20:07:17.393220 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:07:17.393191    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod460c93d5_cd74_436c_8fc4_1d4d7f42348a.slice/crio-c22ebbc98c535d98b4986183f551726f925c3c1a54917a7461bf54d24ff38d1b WatchSource:0}: Error finding container c22ebbc98c535d98b4986183f551726f925c3c1a54917a7461bf54d24ff38d1b: Status 404 returned error can't find the container with id c22ebbc98c535d98b4986183f551726f925c3c1a54917a7461bf54d24ff38d1b
Jul 30 20:07:18.201979 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:18.201922    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8" event={"ID":"01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa","Type":"ContainerStarted","Data":"6081014a20076078060de5b801270d79e041f6346453b2d91f84db9b4c90f8ac"}
Jul 30 20:07:18.202188 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:18.202040    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8"
Jul 30 20:07:18.203495 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:18.203466    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7" event={"ID":"460c93d5-cd74-436c-8fc4-1d4d7f42348a","Type":"ContainerStarted","Data":"391c5389cd75954a00aee7cd12ae2b5ece8307fd0c2d005376a7928d39f78f4f"}
Jul 30 20:07:18.203624 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:18.203507    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7" event={"ID":"460c93d5-cd74-436c-8fc4-1d4d7f42348a","Type":"ContainerStarted","Data":"c22ebbc98c535d98b4986183f551726f925c3c1a54917a7461bf54d24ff38d1b"}
Jul 30 20:07:18.203693 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:18.203622    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7"
Jul 30 20:07:18.219616 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:18.219563    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8" podStartSLOduration=1.405132554 podStartE2EDuration="2.219546376s" totalImagesPullingTime="814.413822ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:07:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:07:18.218890093 +0000 UTC m=+2566.183471169" watchObservedRunningTime="2026-07-30 20:07:18.219546376 +0000 UTC m=+2566.184127459"
Jul 30 20:07:18.237797 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:18.237742    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7" podStartSLOduration=2.237724728 podStartE2EDuration="2.237724728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:07:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:07:18.234245693 +0000 UTC m=+2566.198826776" watchObservedRunningTime="2026-07-30 20:07:18.237724728 +0000 UTC m=+2566.202305811"
Jul 30 20:07:24.215497 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:24.215465    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8"
Jul 30 20:07:29.211397 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:29.211368    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7"
Jul 30 20:07:40.175673 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:40.175640    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:07:40.521471 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:40.521431    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:07:40.919218 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:40.919106    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:07:41.252621 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:41.252580    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:07:44.389421 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:44.389378    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:07:44.509638 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:44.509600    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:07:45.484287 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:45.484256    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6"]
Jul 30 20:07:45.488022 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:45.488001    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6"
Jul 30 20:07:45.490812 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:45.490792    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-ff2f84-serving-cert\""
Jul 30 20:07:45.500165 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:45.500143    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6"]
Jul 30 20:07:45.521383 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:45.521329    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:07:45.610324 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:45.610291    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-744pr\" (UniqueName: \"kubernetes.io/projected/3eb24708-8264-443f-bcc4-b0e90b3843f2-kube-api-access-744pr\") pod \"maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6\" (UID: \"3eb24708-8264-443f-bcc4-b0e90b3843f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6"
Jul 30 20:07:45.610527 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:45.610385    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3eb24708-8264-443f-bcc4-b0e90b3843f2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6\" (UID: \"3eb24708-8264-443f-bcc4-b0e90b3843f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6"
Jul 30 20:07:45.610527 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:45.610512    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3eb24708-8264-443f-bcc4-b0e90b3843f2-maas-api-tls\") pod \"maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6\" (UID: \"3eb24708-8264-443f-bcc4-b0e90b3843f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6"
Jul 30 20:07:45.643648 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:45.643614    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:07:45.711582 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:45.711549    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3eb24708-8264-443f-bcc4-b0e90b3843f2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6\" (UID: \"3eb24708-8264-443f-bcc4-b0e90b3843f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6"
Jul 30 20:07:45.711793 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:45.711602    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3eb24708-8264-443f-bcc4-b0e90b3843f2-maas-api-tls\") pod \"maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6\" (UID: \"3eb24708-8264-443f-bcc4-b0e90b3843f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6"
Jul 30 20:07:45.711793 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:45.711653    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-744pr\" (UniqueName: \"kubernetes.io/projected/3eb24708-8264-443f-bcc4-b0e90b3843f2-kube-api-access-744pr\") pod \"maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6\" (UID: \"3eb24708-8264-443f-bcc4-b0e90b3843f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6"
Jul 30 20:07:45.712426 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:45.712400    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3eb24708-8264-443f-bcc4-b0e90b3843f2-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6\" (UID: \"3eb24708-8264-443f-bcc4-b0e90b3843f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6"
Jul 30 20:07:45.714267 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:45.714242    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3eb24708-8264-443f-bcc4-b0e90b3843f2-maas-api-tls\") pod \"maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6\" (UID: \"3eb24708-8264-443f-bcc4-b0e90b3843f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6"
Jul 30 20:07:45.719351 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:45.719322    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-744pr\" (UniqueName: \"kubernetes.io/projected/3eb24708-8264-443f-bcc4-b0e90b3843f2-kube-api-access-744pr\") pod \"maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6\" (UID: \"3eb24708-8264-443f-bcc4-b0e90b3843f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6"
Jul 30 20:07:45.801751 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:45.801665    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6"
Jul 30 20:07:45.946773 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:45.946735    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6"]
Jul 30 20:07:45.950220 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:07:45.950188    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eb24708_8264_443f_bcc4_b0e90b3843f2.slice/crio-942ded6cfb8da4062954b157109c13fbc5232ccf6dff20fcf42ca6e58691c416 WatchSource:0}: Error finding container 942ded6cfb8da4062954b157109c13fbc5232ccf6dff20fcf42ca6e58691c416: Status 404 returned error can't find the container with id 942ded6cfb8da4062954b157109c13fbc5232ccf6dff20fcf42ca6e58691c416
Jul 30 20:07:46.022414 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:46.022377    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:07:46.119816 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:46.119721    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:07:46.361919 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:46.361887    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6" event={"ID":"3eb24708-8264-443f-bcc4-b0e90b3843f2","Type":"ContainerStarted","Data":"942ded6cfb8da4062954b157109c13fbc5232ccf6dff20fcf42ca6e58691c416"}
Jul 30 20:07:47.369295 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:47.369256    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6" event={"ID":"3eb24708-8264-443f-bcc4-b0e90b3843f2","Type":"ContainerStarted","Data":"b7184a20a285d0a36293a2529414968fea8cd437fbb02d3e7e84b10a2f0915fb"}
Jul 30 20:07:47.369686 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:47.369340    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6"
Jul 30 20:07:47.386478 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:47.386428    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6" podStartSLOduration=2.007422172 podStartE2EDuration="2.386411583s" totalImagesPullingTime="378.989411ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:07:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:07:47.385018032 +0000 UTC m=+2595.349599113" watchObservedRunningTime="2026-07-30 20:07:47.386411583 +0000 UTC m=+2595.350992664"
Jul 30 20:07:53.380145 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:07:53.380113    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6"
Jul 30 20:08:09.318322 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:08:09.318235    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:08:13.490942 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:08:13.490898    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:08:17.154797 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:08:17.154758    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:08:30.922877 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:08:30.922835    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:08:38.602653 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:08:38.602612    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:18.725074 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:18.725030    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:19.569411 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:19.569380    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:19.683036 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:19.683008    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-37abda-74p2t"]
Jul 30 20:09:19.686710 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:19.686686    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-37abda-74p2t"
Jul 30 20:09:19.689224 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:19.689197    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7t7kk\""
Jul 30 20:09:19.694505 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:19.694478    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-37abda-74p2t"]
Jul 30 20:09:19.822933 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:19.822838    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf724680-5158-4a05-883e-7472c9ab53bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-37abda-74p2t\" (UID: \"cf724680-5158-4a05-883e-7472c9ab53bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-37abda-74p2t"
Jul 30 20:09:19.924422 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:19.924388    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf724680-5158-4a05-883e-7472c9ab53bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-37abda-74p2t\" (UID: \"cf724680-5158-4a05-883e-7472c9ab53bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-37abda-74p2t"
Jul 30 20:09:19.925209 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:19.925186    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf724680-5158-4a05-883e-7472c9ab53bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-37abda-74p2t\" (UID: \"cf724680-5158-4a05-883e-7472c9ab53bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-37abda-74p2t"
Jul 30 20:09:20.000781 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:20.000749    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-37abda-74p2t"
Jul 30 20:09:20.145091 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:20.145055    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-37abda-74p2t"]
Jul 30 20:09:20.146095 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:09:20.146069    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf724680_5158_4a05_883e_7472c9ab53bc.slice/crio-dca0bde37bcdea13756c15786cbc574156a41948b1ab80c3f549004ab9f8aff5 WatchSource:0}: Error finding container dca0bde37bcdea13756c15786cbc574156a41948b1ab80c3f549004ab9f8aff5: Status 404 returned error can't find the container with id dca0bde37bcdea13756c15786cbc574156a41948b1ab80c3f549004ab9f8aff5
Jul 30 20:09:20.900739 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:20.900707    2554 generic.go:356] "Generic (PLEG): container finished" podID="cf724680-5158-4a05-883e-7472c9ab53bc" containerID="ca575139a28c49b33429055487c4707e0ff965ce580a3c27847255b01ca88db1" exitCode=0
Jul 30 20:09:20.901178 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:20.900795    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-37abda-74p2t" event={"ID":"cf724680-5158-4a05-883e-7472c9ab53bc","Type":"ContainerDied","Data":"ca575139a28c49b33429055487c4707e0ff965ce580a3c27847255b01ca88db1"}
Jul 30 20:09:20.901178 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:20.900864    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-37abda-74p2t" event={"ID":"cf724680-5158-4a05-883e-7472c9ab53bc","Type":"ContainerStarted","Data":"dca0bde37bcdea13756c15786cbc574156a41948b1ab80c3f549004ab9f8aff5"}
Jul 30 20:09:22.046965 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:22.046930    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-37abda-74p2t"
Jul 30 20:09:22.145691 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:22.145654    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf724680-5158-4a05-883e-7472c9ab53bc-maas-api-service-ca\") pod \"cf724680-5158-4a05-883e-7472c9ab53bc\" (UID: \"cf724680-5158-4a05-883e-7472c9ab53bc\") "
Jul 30 20:09:22.146071 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:22.146049    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf724680-5158-4a05-883e-7472c9ab53bc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cf724680-5158-4a05-883e-7472c9ab53bc" (UID: "cf724680-5158-4a05-883e-7472c9ab53bc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:09:22.247431 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:22.247397    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf724680-5158-4a05-883e-7472c9ab53bc-maas-api-service-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:09:22.913868 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:22.913841    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-37abda-74p2t"
Jul 30 20:09:22.914072 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:22.913879    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-37abda-74p2t" event={"ID":"cf724680-5158-4a05-883e-7472c9ab53bc","Type":"ContainerDied","Data":"dca0bde37bcdea13756c15786cbc574156a41948b1ab80c3f549004ab9f8aff5"}
Jul 30 20:09:22.914072 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:22.913927    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dca0bde37bcdea13756c15786cbc574156a41948b1ab80c3f549004ab9f8aff5"
Jul 30 20:09:30.164650 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.164565    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8"]
Jul 30 20:09:30.165110 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.164848    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8" podUID="01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa" containerName="maas-api" containerID="cri-o://6081014a20076078060de5b801270d79e041f6346453b2d91f84db9b4c90f8ac" gracePeriod=30
Jul 30 20:09:30.179946 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.179910    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:30.215993 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.215777    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7"]
Jul 30 20:09:30.216158 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.216043    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7" podUID="460c93d5-cd74-436c-8fc4-1d4d7f42348a" containerName="payload-processing" containerID="cri-o://391c5389cd75954a00aee7cd12ae2b5ece8307fd0c2d005376a7928d39f78f4f" gracePeriod=30
Jul 30 20:09:30.290268 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.290237    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:30.395979 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.395910    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:30.465187 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.464994    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-37abda-74p2t"]
Jul 30 20:09:30.467652 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.467618    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:30.468871 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.468851    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-37abda-74p2t"]
Jul 30 20:09:30.474426 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.474409    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8"
Jul 30 20:09:30.488487 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.488461    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf724680-5158-4a05-883e-7472c9ab53bc" path="/var/lib/kubelet/pods/cf724680-5158-4a05-883e-7472c9ab53bc/volumes"
Jul 30 20:09:30.505529 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.505504    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7"
Jul 30 20:09:30.523911 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.523882    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa-openshift-service-ca\") pod \"01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa\" (UID: \"01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa\") "
Jul 30 20:09:30.524087 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.523943    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/460c93d5-cd74-436c-8fc4-1d4d7f42348a-plugins-config-volume\") pod \"460c93d5-cd74-436c-8fc4-1d4d7f42348a\" (UID: \"460c93d5-cd74-436c-8fc4-1d4d7f42348a\") "
Jul 30 20:09:30.524087 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.523998    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa-maas-api-tls\") pod \"01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa\" (UID: \"01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa\") "
Jul 30 20:09:30.524087 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.524045    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xq4gz\" (UniqueName: \"kubernetes.io/projected/460c93d5-cd74-436c-8fc4-1d4d7f42348a-kube-api-access-xq4gz\") pod \"460c93d5-cd74-436c-8fc4-1d4d7f42348a\" (UID: \"460c93d5-cd74-436c-8fc4-1d4d7f42348a\") "
Jul 30 20:09:30.524349 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.524104    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7x78\" (UniqueName: \"kubernetes.io/projected/01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa-kube-api-access-t7x78\") pod \"01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa\" (UID: \"01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa\") "
Jul 30 20:09:30.524408 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.524363    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa" (UID: "01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:09:30.524487 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.524469    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa-openshift-service-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:09:30.525419 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.525394    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/460c93d5-cd74-436c-8fc4-1d4d7f42348a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "460c93d5-cd74-436c-8fc4-1d4d7f42348a" (UID: "460c93d5-cd74-436c-8fc4-1d4d7f42348a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:09:30.526873 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.526839    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa" (UID: "01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:09:30.527589 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.527559    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/460c93d5-cd74-436c-8fc4-1d4d7f42348a-kube-api-access-xq4gz" (OuterVolumeSpecName: "kube-api-access-xq4gz") pod "460c93d5-cd74-436c-8fc4-1d4d7f42348a" (UID: "460c93d5-cd74-436c-8fc4-1d4d7f42348a"). InnerVolumeSpecName "kube-api-access-xq4gz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:09:30.527693 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.527643    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa-kube-api-access-t7x78" (OuterVolumeSpecName: "kube-api-access-t7x78") pod "01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa" (UID: "01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa"). InnerVolumeSpecName "kube-api-access-t7x78". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:09:30.625150 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.625120    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7x78\" (UniqueName: \"kubernetes.io/projected/01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa-kube-api-access-t7x78\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:09:30.625150 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.625146    2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/460c93d5-cd74-436c-8fc4-1d4d7f42348a-plugins-config-volume\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:09:30.625150 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.625156    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa-maas-api-tls\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:09:30.625379 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.625165    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xq4gz\" (UniqueName: \"kubernetes.io/projected/460c93d5-cd74-436c-8fc4-1d4d7f42348a-kube-api-access-xq4gz\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:09:30.858702 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.858663    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:30.963712 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.963677    2554 generic.go:356] "Generic (PLEG): container finished" podID="460c93d5-cd74-436c-8fc4-1d4d7f42348a" containerID="391c5389cd75954a00aee7cd12ae2b5ece8307fd0c2d005376a7928d39f78f4f" exitCode=0
Jul 30 20:09:30.963909 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.963726    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7"
Jul 30 20:09:30.963909 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.963761    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7" event={"ID":"460c93d5-cd74-436c-8fc4-1d4d7f42348a","Type":"ContainerDied","Data":"391c5389cd75954a00aee7cd12ae2b5ece8307fd0c2d005376a7928d39f78f4f"}
Jul 30 20:09:30.963909 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.963816    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7" event={"ID":"460c93d5-cd74-436c-8fc4-1d4d7f42348a","Type":"ContainerDied","Data":"c22ebbc98c535d98b4986183f551726f925c3c1a54917a7461bf54d24ff38d1b"}
Jul 30 20:09:30.963909 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.963847    2554 scope.go:117] "RemoveContainer" containerID="391c5389cd75954a00aee7cd12ae2b5ece8307fd0c2d005376a7928d39f78f4f"
Jul 30 20:09:30.965160 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.965138    2554 generic.go:356] "Generic (PLEG): container finished" podID="01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa" containerID="6081014a20076078060de5b801270d79e041f6346453b2d91f84db9b4c90f8ac" exitCode=0
Jul 30 20:09:30.965271 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.965177    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8" event={"ID":"01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa","Type":"ContainerDied","Data":"6081014a20076078060de5b801270d79e041f6346453b2d91f84db9b4c90f8ac"}
Jul 30 20:09:30.965271 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.965189    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8"
Jul 30 20:09:30.965271 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.965211    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8" event={"ID":"01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa","Type":"ContainerDied","Data":"4a11b5f1b17b9820bd3e7328d637a66949772801e1633628765c8b1b5130e35b"}
Jul 30 20:09:30.974909 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.974891    2554 scope.go:117] "RemoveContainer" containerID="391c5389cd75954a00aee7cd12ae2b5ece8307fd0c2d005376a7928d39f78f4f"
Jul 30 20:09:30.975194 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:09:30.975170    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"391c5389cd75954a00aee7cd12ae2b5ece8307fd0c2d005376a7928d39f78f4f\": container with ID starting with 391c5389cd75954a00aee7cd12ae2b5ece8307fd0c2d005376a7928d39f78f4f not found: ID does not exist" containerID="391c5389cd75954a00aee7cd12ae2b5ece8307fd0c2d005376a7928d39f78f4f"
Jul 30 20:09:30.975299 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.975201    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"391c5389cd75954a00aee7cd12ae2b5ece8307fd0c2d005376a7928d39f78f4f"} err="failed to get container status \"391c5389cd75954a00aee7cd12ae2b5ece8307fd0c2d005376a7928d39f78f4f\": rpc error: code = NotFound desc = could not find container \"391c5389cd75954a00aee7cd12ae2b5ece8307fd0c2d005376a7928d39f78f4f\": container with ID starting with 391c5389cd75954a00aee7cd12ae2b5ece8307fd0c2d005376a7928d39f78f4f not found: ID does not exist"
Jul 30 20:09:30.975299 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.975218    2554 scope.go:117] "RemoveContainer" containerID="6081014a20076078060de5b801270d79e041f6346453b2d91f84db9b4c90f8ac"
Jul 30 20:09:30.983467 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.983449    2554 scope.go:117] "RemoveContainer" containerID="6081014a20076078060de5b801270d79e041f6346453b2d91f84db9b4c90f8ac"
Jul 30 20:09:30.983716 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:09:30.983698    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6081014a20076078060de5b801270d79e041f6346453b2d91f84db9b4c90f8ac\": container with ID starting with 6081014a20076078060de5b801270d79e041f6346453b2d91f84db9b4c90f8ac not found: ID does not exist" containerID="6081014a20076078060de5b801270d79e041f6346453b2d91f84db9b4c90f8ac"
Jul 30 20:09:30.983793 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.983730    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6081014a20076078060de5b801270d79e041f6346453b2d91f84db9b4c90f8ac"} err="failed to get container status \"6081014a20076078060de5b801270d79e041f6346453b2d91f84db9b4c90f8ac\": rpc error: code = NotFound desc = could not find container \"6081014a20076078060de5b801270d79e041f6346453b2d91f84db9b4c90f8ac\": container with ID starting with 6081014a20076078060de5b801270d79e041f6346453b2d91f84db9b4c90f8ac not found: ID does not exist"
Jul 30 20:09:30.988349 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.988329    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7"]
Jul 30 20:09:30.992517 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:30.992497    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-37abda-686b856cbb-hhfc7"]
Jul 30 20:09:31.004566 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:31.004545    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8"]
Jul 30 20:09:31.009072 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:31.009051    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-37abda-558bcc948d-nbgq8"]
Jul 30 20:09:32.486411 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:32.486369    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa" path="/var/lib/kubelet/pods/01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa/volumes"
Jul 30 20:09:32.486937 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:32.486916    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="460c93d5-cd74-436c-8fc4-1d4d7f42348a" path="/var/lib/kubelet/pods/460c93d5-cd74-436c-8fc4-1d4d7f42348a/volumes"
Jul 30 20:09:32.750025 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:32.749923    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/1.log"
Jul 30 20:09:32.754651 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:32.754625    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 20:09:32.758841 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:32.758824    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/1.log"
Jul 30 20:09:32.763198 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:32.763179    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 20:09:36.596633 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:36.596598    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:36.694333 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:36.694297    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:36.863133 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:36.863043    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:37.095155 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:37.095114    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:37.204370 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:37.204294    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff2f84-nf4kb"]
Jul 30 20:09:37.204710 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:37.204697    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="460c93d5-cd74-436c-8fc4-1d4d7f42348a" containerName="payload-processing"
Jul 30 20:09:37.204753 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:37.204712    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="460c93d5-cd74-436c-8fc4-1d4d7f42348a" containerName="payload-processing"
Jul 30 20:09:37.204753 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:37.204723    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa" containerName="maas-api"
Jul 30 20:09:37.204753 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:37.204729    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa" containerName="maas-api"
Jul 30 20:09:37.204753 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:37.204737    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf724680-5158-4a05-883e-7472c9ab53bc" containerName="revoke-keys"
Jul 30 20:09:37.204753 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:37.204743    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf724680-5158-4a05-883e-7472c9ab53bc" containerName="revoke-keys"
Jul 30 20:09:37.204905 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:37.204799    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="460c93d5-cd74-436c-8fc4-1d4d7f42348a" containerName="payload-processing"
Jul 30 20:09:37.204905 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:37.204809    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="01a9b07c-fe8c-4d58-aa7a-6a7ecd0af4aa" containerName="maas-api"
Jul 30 20:09:37.204905 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:37.204817    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf724680-5158-4a05-883e-7472c9ab53bc" containerName="revoke-keys"
Jul 30 20:09:37.209500 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:37.209475    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff2f84-nf4kb"
Jul 30 20:09:37.215650 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:37.215626    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7t7kk\""
Jul 30 20:09:37.215868 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:37.215841    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff2f84-nf4kb"]
Jul 30 20:09:37.295361 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:37.295324    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfbb51f5-37cd-44bf-b3c9-ef2c858e7ac4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ff2f84-nf4kb\" (UID: \"cfbb51f5-37cd-44bf-b3c9-ef2c858e7ac4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff2f84-nf4kb"
Jul 30 20:09:37.396874 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:37.396835    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfbb51f5-37cd-44bf-b3c9-ef2c858e7ac4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ff2f84-nf4kb\" (UID: \"cfbb51f5-37cd-44bf-b3c9-ef2c858e7ac4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff2f84-nf4kb"
Jul 30 20:09:37.397568 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:37.397532    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfbb51f5-37cd-44bf-b3c9-ef2c858e7ac4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ff2f84-nf4kb\" (UID: \"cfbb51f5-37cd-44bf-b3c9-ef2c858e7ac4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff2f84-nf4kb"
Jul 30 20:09:37.527705 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:37.527670    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff2f84-nf4kb"
Jul 30 20:09:37.667892 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:37.667867    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff2f84-nf4kb"]
Jul 30 20:09:37.669900 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:09:37.669869    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfbb51f5_37cd_44bf_b3c9_ef2c858e7ac4.slice/crio-fdcd74497a70becc4e58bd7bb3dba9d0e7e460a9456dca9e71094b74e18e6977 WatchSource:0}: Error finding container fdcd74497a70becc4e58bd7bb3dba9d0e7e460a9456dca9e71094b74e18e6977: Status 404 returned error can't find the container with id fdcd74497a70becc4e58bd7bb3dba9d0e7e460a9456dca9e71094b74e18e6977
Jul 30 20:09:38.007299 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:38.007265    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff2f84-nf4kb" event={"ID":"cfbb51f5-37cd-44bf-b3c9-ef2c858e7ac4","Type":"ContainerStarted","Data":"fdcd74497a70becc4e58bd7bb3dba9d0e7e460a9456dca9e71094b74e18e6977"}
Jul 30 20:09:39.015495 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:39.015462    2554 generic.go:356] "Generic (PLEG): container finished" podID="cfbb51f5-37cd-44bf-b3c9-ef2c858e7ac4" containerID="0af7a360a19efcdc920a604fc226ab9eca2b25b9a0f23a1652d8e73961b0e982" exitCode=0
Jul 30 20:09:39.016054 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:39.015558    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff2f84-nf4kb" event={"ID":"cfbb51f5-37cd-44bf-b3c9-ef2c858e7ac4","Type":"ContainerDied","Data":"0af7a360a19efcdc920a604fc226ab9eca2b25b9a0f23a1652d8e73961b0e982"}
Jul 30 20:09:40.157404 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:40.157379    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff2f84-nf4kb"
Jul 30 20:09:40.225933 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:40.225901    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfbb51f5-37cd-44bf-b3c9-ef2c858e7ac4-maas-api-service-ca\") pod \"cfbb51f5-37cd-44bf-b3c9-ef2c858e7ac4\" (UID: \"cfbb51f5-37cd-44bf-b3c9-ef2c858e7ac4\") "
Jul 30 20:09:40.226254 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:40.226233    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfbb51f5-37cd-44bf-b3c9-ef2c858e7ac4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cfbb51f5-37cd-44bf-b3c9-ef2c858e7ac4" (UID: "cfbb51f5-37cd-44bf-b3c9-ef2c858e7ac4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:09:40.327668 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:40.327591    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfbb51f5-37cd-44bf-b3c9-ef2c858e7ac4-maas-api-service-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:09:41.030554 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:41.030520    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff2f84-nf4kb"
Jul 30 20:09:41.030742 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:41.030552    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff2f84-nf4kb" event={"ID":"cfbb51f5-37cd-44bf-b3c9-ef2c858e7ac4","Type":"ContainerDied","Data":"fdcd74497a70becc4e58bd7bb3dba9d0e7e460a9456dca9e71094b74e18e6977"}
Jul 30 20:09:41.030742 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:41.030610    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fdcd74497a70becc4e58bd7bb3dba9d0e7e460a9456dca9e71094b74e18e6977"
Jul 30 20:09:47.772919 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:47.772884    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6"]
Jul 30 20:09:47.773419 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:47.773168    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6" podUID="3eb24708-8264-443f-bcc4-b0e90b3843f2" containerName="maas-api" containerID="cri-o://b7184a20a285d0a36293a2529414968fea8cd437fbb02d3e7e84b10a2f0915fb" gracePeriod=30
Jul 30 20:09:47.782419 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:47.782385    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:47.854671 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:47.854634    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:47.911238 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:47.911193    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:47.949715 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:47.949684    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff2f84-nf4kb"]
Jul 30 20:09:47.955846 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:47.955811    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff2f84-nf4kb"]
Jul 30 20:09:47.966500 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:47.966445    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:48.040057 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:48.040019    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6"
Jul 30 20:09:48.075919 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:48.075887    2554 generic.go:356] "Generic (PLEG): container finished" podID="3eb24708-8264-443f-bcc4-b0e90b3843f2" containerID="b7184a20a285d0a36293a2529414968fea8cd437fbb02d3e7e84b10a2f0915fb" exitCode=0
Jul 30 20:09:48.076142 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:48.075941    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6" event={"ID":"3eb24708-8264-443f-bcc4-b0e90b3843f2","Type":"ContainerDied","Data":"b7184a20a285d0a36293a2529414968fea8cd437fbb02d3e7e84b10a2f0915fb"}
Jul 30 20:09:48.076142 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:48.075999    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6" event={"ID":"3eb24708-8264-443f-bcc4-b0e90b3843f2","Type":"ContainerDied","Data":"942ded6cfb8da4062954b157109c13fbc5232ccf6dff20fcf42ca6e58691c416"}
Jul 30 20:09:48.076142 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:48.076022    2554 scope.go:117] "RemoveContainer" containerID="b7184a20a285d0a36293a2529414968fea8cd437fbb02d3e7e84b10a2f0915fb"
Jul 30 20:09:48.076142 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:48.075942    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6"
Jul 30 20:09:48.093761 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:48.093738    2554 scope.go:117] "RemoveContainer" containerID="b7184a20a285d0a36293a2529414968fea8cd437fbb02d3e7e84b10a2f0915fb"
Jul 30 20:09:48.094084 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:09:48.094052    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7184a20a285d0a36293a2529414968fea8cd437fbb02d3e7e84b10a2f0915fb\": container with ID starting with b7184a20a285d0a36293a2529414968fea8cd437fbb02d3e7e84b10a2f0915fb not found: ID does not exist" containerID="b7184a20a285d0a36293a2529414968fea8cd437fbb02d3e7e84b10a2f0915fb"
Jul 30 20:09:48.094146 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:48.094096    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7184a20a285d0a36293a2529414968fea8cd437fbb02d3e7e84b10a2f0915fb"} err="failed to get container status \"b7184a20a285d0a36293a2529414968fea8cd437fbb02d3e7e84b10a2f0915fb\": rpc error: code = NotFound desc = could not find container \"b7184a20a285d0a36293a2529414968fea8cd437fbb02d3e7e84b10a2f0915fb\": container with ID starting with b7184a20a285d0a36293a2529414968fea8cd437fbb02d3e7e84b10a2f0915fb not found: ID does not exist"
Jul 30 20:09:48.101568 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:48.101533    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-744pr\" (UniqueName: \"kubernetes.io/projected/3eb24708-8264-443f-bcc4-b0e90b3843f2-kube-api-access-744pr\") pod \"3eb24708-8264-443f-bcc4-b0e90b3843f2\" (UID: \"3eb24708-8264-443f-bcc4-b0e90b3843f2\") "
Jul 30 20:09:48.101697 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:48.101582    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3eb24708-8264-443f-bcc4-b0e90b3843f2-openshift-service-ca\") pod \"3eb24708-8264-443f-bcc4-b0e90b3843f2\" (UID: \"3eb24708-8264-443f-bcc4-b0e90b3843f2\") "
Jul 30 20:09:48.101769 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:48.101717    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3eb24708-8264-443f-bcc4-b0e90b3843f2-maas-api-tls\") pod \"3eb24708-8264-443f-bcc4-b0e90b3843f2\" (UID: \"3eb24708-8264-443f-bcc4-b0e90b3843f2\") "
Jul 30 20:09:48.102120 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:48.102091    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3eb24708-8264-443f-bcc4-b0e90b3843f2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3eb24708-8264-443f-bcc4-b0e90b3843f2" (UID: "3eb24708-8264-443f-bcc4-b0e90b3843f2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:09:48.103862 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:48.103827    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3eb24708-8264-443f-bcc4-b0e90b3843f2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3eb24708-8264-443f-bcc4-b0e90b3843f2" (UID: "3eb24708-8264-443f-bcc4-b0e90b3843f2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:09:48.103942 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:48.103883    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eb24708-8264-443f-bcc4-b0e90b3843f2-kube-api-access-744pr" (OuterVolumeSpecName: "kube-api-access-744pr") pod "3eb24708-8264-443f-bcc4-b0e90b3843f2" (UID: "3eb24708-8264-443f-bcc4-b0e90b3843f2"). InnerVolumeSpecName "kube-api-access-744pr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:09:48.203408 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:48.203374    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3eb24708-8264-443f-bcc4-b0e90b3843f2-maas-api-tls\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:09:48.203408 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:48.203402    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-744pr\" (UniqueName: \"kubernetes.io/projected/3eb24708-8264-443f-bcc4-b0e90b3843f2-kube-api-access-744pr\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:09:48.203408 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:48.203413    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3eb24708-8264-443f-bcc4-b0e90b3843f2-openshift-service-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:09:48.321548 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:48.321455    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:48.399475 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:48.399443    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6"]
Jul 30 20:09:48.403161 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:48.403136    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ff2f84-64d4b7865-2zfc6"]
Jul 30 20:09:48.486807 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:48.486774    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3eb24708-8264-443f-bcc4-b0e90b3843f2" path="/var/lib/kubelet/pods/3eb24708-8264-443f-bcc4-b0e90b3843f2/volumes"
Jul 30 20:09:48.487172 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:48.487156    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfbb51f5-37cd-44bf-b3c9-ef2c858e7ac4" path="/var/lib/kubelet/pods/cfbb51f5-37cd-44bf-b3c9-ef2c858e7ac4/volumes"
Jul 30 20:09:50.784755 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:50.784718    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:54.022824 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:54.022780    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:54.072142 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:54.072097    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:54.309891 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:54.309794    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:54.535434 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:54.535396    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:54.892380 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:54.892345    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:55.199206 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:55.199124    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:55.436163 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:55.436124    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:57.908465 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:57.908430    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:58.007327 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:58.007294    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:58.986284 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:58.986251    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk"]
Jul 30 20:09:58.986692 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:58.986656    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3eb24708-8264-443f-bcc4-b0e90b3843f2" containerName="maas-api"
Jul 30 20:09:58.986692 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:58.986667    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="3eb24708-8264-443f-bcc4-b0e90b3843f2" containerName="maas-api"
Jul 30 20:09:58.986692 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:58.986677    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfbb51f5-37cd-44bf-b3c9-ef2c858e7ac4" containerName="revoke-keys"
Jul 30 20:09:58.986692 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:58.986683    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfbb51f5-37cd-44bf-b3c9-ef2c858e7ac4" containerName="revoke-keys"
Jul 30 20:09:58.986852 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:58.986748    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfbb51f5-37cd-44bf-b3c9-ef2c858e7ac4" containerName="revoke-keys"
Jul 30 20:09:58.986852 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:58.986759    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="3eb24708-8264-443f-bcc4-b0e90b3843f2" containerName="maas-api"
Jul 30 20:09:58.991005 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:58.990983    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk"
Jul 30 20:09:58.993565 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:58.993544    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-ec1ee8-serving-cert\""
Jul 30 20:09:59.002274 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:59.002249    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk"]
Jul 30 20:09:59.020389 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:59.020358    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:59.105543 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:59.105514    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2c8m\" (UniqueName: \"kubernetes.io/projected/0ad293db-6103-4fda-aae3-198c127d55f2-kube-api-access-w2c8m\") pod \"maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk\" (UID: \"0ad293db-6103-4fda-aae3-198c127d55f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk"
Jul 30 20:09:59.105543 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:59.105547    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad293db-6103-4fda-aae3-198c127d55f2-openshift-service-ca\") pod \"maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk\" (UID: \"0ad293db-6103-4fda-aae3-198c127d55f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk"
Jul 30 20:09:59.105808 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:59.105570    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ad293db-6103-4fda-aae3-198c127d55f2-maas-api-tls\") pod \"maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk\" (UID: \"0ad293db-6103-4fda-aae3-198c127d55f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk"
Jul 30 20:09:59.207059 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:59.207025    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2c8m\" (UniqueName: \"kubernetes.io/projected/0ad293db-6103-4fda-aae3-198c127d55f2-kube-api-access-w2c8m\") pod \"maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk\" (UID: \"0ad293db-6103-4fda-aae3-198c127d55f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk"
Jul 30 20:09:59.207242 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:59.207064    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad293db-6103-4fda-aae3-198c127d55f2-openshift-service-ca\") pod \"maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk\" (UID: \"0ad293db-6103-4fda-aae3-198c127d55f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk"
Jul 30 20:09:59.207242 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:59.207086    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ad293db-6103-4fda-aae3-198c127d55f2-maas-api-tls\") pod \"maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk\" (UID: \"0ad293db-6103-4fda-aae3-198c127d55f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk"
Jul 30 20:09:59.207322 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:09:59.207238    2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-serving-cert: secret "maas-api-e2e-api-a-ec1ee8-serving-cert" not found
Jul 30 20:09:59.207322 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:09:59.207311    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ad293db-6103-4fda-aae3-198c127d55f2-maas-api-tls podName:0ad293db-6103-4fda-aae3-198c127d55f2 nodeName:}" failed. No retries permitted until 2026-07-30 20:09:59.707295525 +0000 UTC m=+2727.671876584 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0ad293db-6103-4fda-aae3-198c127d55f2-maas-api-tls") pod "maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk" (UID: "0ad293db-6103-4fda-aae3-198c127d55f2") : secret "maas-api-e2e-api-a-ec1ee8-serving-cert" not found
Jul 30 20:09:59.207809 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:59.207790    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad293db-6103-4fda-aae3-198c127d55f2-openshift-service-ca\") pod \"maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk\" (UID: \"0ad293db-6103-4fda-aae3-198c127d55f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk"
Jul 30 20:09:59.216035 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:59.215948    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2c8m\" (UniqueName: \"kubernetes.io/projected/0ad293db-6103-4fda-aae3-198c127d55f2-kube-api-access-w2c8m\") pod \"maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk\" (UID: \"0ad293db-6103-4fda-aae3-198c127d55f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk"
Jul 30 20:09:59.263462 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:59.263370    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:59.601244 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:59.601157    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:59.704456 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:59.704418    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:09:59.712681 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:59.712639    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ad293db-6103-4fda-aae3-198c127d55f2-maas-api-tls\") pod \"maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk\" (UID: \"0ad293db-6103-4fda-aae3-198c127d55f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk"
Jul 30 20:09:59.715107 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:59.715085    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ad293db-6103-4fda-aae3-198c127d55f2-maas-api-tls\") pod \"maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk\" (UID: \"0ad293db-6103-4fda-aae3-198c127d55f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk"
Jul 30 20:09:59.904975 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:09:59.904868    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk"
Jul 30 20:10:00.050045 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:00.050007    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk"]
Jul 30 20:10:00.050464 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:10:00.050366    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ad293db_6103_4fda_aae3_198c127d55f2.slice/crio-3a4f3a429633314dedd616ea11a7941db8c96d243c4e0f514545ad479253030d WatchSource:0}: Error finding container 3a4f3a429633314dedd616ea11a7941db8c96d243c4e0f514545ad479253030d: Status 404 returned error can't find the container with id 3a4f3a429633314dedd616ea11a7941db8c96d243c4e0f514545ad479253030d
Jul 30 20:10:00.051802 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:00.051776    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 20:10:00.150315 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:00.150279    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk" event={"ID":"0ad293db-6103-4fda-aae3-198c127d55f2","Type":"ContainerStarted","Data":"3a4f3a429633314dedd616ea11a7941db8c96d243c4e0f514545ad479253030d"}
Jul 30 20:10:01.157334 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:01.157300    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk" event={"ID":"0ad293db-6103-4fda-aae3-198c127d55f2","Type":"ContainerStarted","Data":"0491c2c72e86899b713e034eba1a17a4af17d1b49c45821479a110e53c75a74d"}
Jul 30 20:10:01.157795 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:01.157480    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk"
Jul 30 20:10:01.177202 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:01.177155    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk" podStartSLOduration=2.679044645 podStartE2EDuration="3.17713988s" totalImagesPullingTime="498.095235ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:09:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:10:01.175733391 +0000 UTC m=+2729.140314470" watchObservedRunningTime="2026-07-30 20:10:01.17713988 +0000 UTC m=+2729.141720986"
Jul 30 20:10:07.169474 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:07.169445    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk"
Jul 30 20:10:22.905924 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:22.905889    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:10:23.367103 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:23.367067    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:10:23.675520 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:23.675434    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:10:23.851198 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:23.851162    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:10:26.980645 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:26.980606    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:10:27.081579 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:27.081541    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:10:28.079903 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:28.079872    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg"]
Jul 30 20:10:28.083544 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:28.083524    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg"
Jul 30 20:10:28.085726 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:28.085704    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-18459c-serving-cert\""
Jul 30 20:10:28.094074 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:28.094045    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg"]
Jul 30 20:10:28.109040 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:28.108996    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:10:28.179146 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:28.179112    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plrzm\" (UniqueName: \"kubernetes.io/projected/6f4d75d8-6d9b-4c69-85ae-6951b669228f-kube-api-access-plrzm\") pod \"maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg\" (UID: \"6f4d75d8-6d9b-4c69-85ae-6951b669228f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg"
Jul 30 20:10:28.179340 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:28.179169    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f4d75d8-6d9b-4c69-85ae-6951b669228f-openshift-service-ca\") pod \"maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg\" (UID: \"6f4d75d8-6d9b-4c69-85ae-6951b669228f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg"
Jul 30 20:10:28.179340 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:28.179293    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f4d75d8-6d9b-4c69-85ae-6951b669228f-maas-api-tls\") pod \"maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg\" (UID: \"6f4d75d8-6d9b-4c69-85ae-6951b669228f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg"
Jul 30 20:10:28.280366 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:28.280331    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f4d75d8-6d9b-4c69-85ae-6951b669228f-openshift-service-ca\") pod \"maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg\" (UID: \"6f4d75d8-6d9b-4c69-85ae-6951b669228f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg"
Jul 30 20:10:28.280543 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:28.280384    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f4d75d8-6d9b-4c69-85ae-6951b669228f-maas-api-tls\") pod \"maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg\" (UID: \"6f4d75d8-6d9b-4c69-85ae-6951b669228f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg"
Jul 30 20:10:28.280543 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:28.280448    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plrzm\" (UniqueName: \"kubernetes.io/projected/6f4d75d8-6d9b-4c69-85ae-6951b669228f-kube-api-access-plrzm\") pod \"maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg\" (UID: \"6f4d75d8-6d9b-4c69-85ae-6951b669228f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg"
Jul 30 20:10:28.280620 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:10:28.280600    2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-serving-cert: secret "maas-api-e2e-api-b-18459c-serving-cert" not found
Jul 30 20:10:28.280703 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:10:28.280689    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f4d75d8-6d9b-4c69-85ae-6951b669228f-maas-api-tls podName:6f4d75d8-6d9b-4c69-85ae-6951b669228f nodeName:}" failed. No retries permitted until 2026-07-30 20:10:28.780666491 +0000 UTC m=+2756.745247556 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6f4d75d8-6d9b-4c69-85ae-6951b669228f-maas-api-tls") pod "maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg" (UID: "6f4d75d8-6d9b-4c69-85ae-6951b669228f") : secret "maas-api-e2e-api-b-18459c-serving-cert" not found
Jul 30 20:10:28.280977 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:28.280937    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f4d75d8-6d9b-4c69-85ae-6951b669228f-openshift-service-ca\") pod \"maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg\" (UID: \"6f4d75d8-6d9b-4c69-85ae-6951b669228f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg"
Jul 30 20:10:28.288856 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:28.288829    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plrzm\" (UniqueName: \"kubernetes.io/projected/6f4d75d8-6d9b-4c69-85ae-6951b669228f-kube-api-access-plrzm\") pod \"maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg\" (UID: \"6f4d75d8-6d9b-4c69-85ae-6951b669228f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg"
Jul 30 20:10:28.420602 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:28.420484    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:10:28.698203 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:28.698093    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:10:28.785467 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:28.785433    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f4d75d8-6d9b-4c69-85ae-6951b669228f-maas-api-tls\") pod \"maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg\" (UID: \"6f4d75d8-6d9b-4c69-85ae-6951b669228f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg"
Jul 30 20:10:28.787854 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:28.787822    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f4d75d8-6d9b-4c69-85ae-6951b669228f-maas-api-tls\") pod \"maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg\" (UID: \"6f4d75d8-6d9b-4c69-85ae-6951b669228f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg"
Jul 30 20:10:28.798715 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:28.798686    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:10:28.996910 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:28.996876    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg"
Jul 30 20:10:29.137998 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:10:29.137921    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f4d75d8_6d9b_4c69_85ae_6951b669228f.slice/crio-0345bf30664a13d0213d8252c0facbaa07e6f8faec58e1132b0ab4e71fd0f6b4 WatchSource:0}: Error finding container 0345bf30664a13d0213d8252c0facbaa07e6f8faec58e1132b0ab4e71fd0f6b4: Status 404 returned error can't find the container with id 0345bf30664a13d0213d8252c0facbaa07e6f8faec58e1132b0ab4e71fd0f6b4
Jul 30 20:10:29.138407 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:29.138304    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg"]
Jul 30 20:10:29.329521 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:29.329438    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg" event={"ID":"6f4d75d8-6d9b-4c69-85ae-6951b669228f","Type":"ContainerStarted","Data":"0345bf30664a13d0213d8252c0facbaa07e6f8faec58e1132b0ab4e71fd0f6b4"}
Jul 30 20:10:30.337834 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:30.337793    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg" event={"ID":"6f4d75d8-6d9b-4c69-85ae-6951b669228f","Type":"ContainerStarted","Data":"bb7cd8761dce8855aab17993173a7f2105e980a096f8b69e2a515e93eec8b034"}
Jul 30 20:10:30.338267 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:30.337877    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg"
Jul 30 20:10:36.351066 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:36.351037    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg"
Jul 30 20:10:36.370206 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:36.370150    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg" podStartSLOduration=8.029378144 podStartE2EDuration="8.370135033s" totalImagesPullingTime="340.756889ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:10:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:10:30.358016417 +0000 UTC m=+2758.322597499" watchObservedRunningTime="2026-07-30 20:10:36.370135033 +0000 UTC m=+2764.334716115"
Jul 30 20:10:51.984153 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:51.984119    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:10:53.386849 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:53.386786    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:10:53.487796 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:53.487689    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:10:54.859679 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:54.859581    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ec1ee8-8c697"]
Jul 30 20:10:54.863391 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:54.863367    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ec1ee8-8c697"
Jul 30 20:10:54.865768 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:54.865745    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7t7kk\""
Jul 30 20:10:54.871220 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:54.871195    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ec1ee8-8c697"]
Jul 30 20:10:54.945947 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:54.945917    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cc146c0-1313-4905-98d8-27bb4b43eb4d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ec1ee8-8c697\" (UID: \"4cc146c0-1313-4905-98d8-27bb4b43eb4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ec1ee8-8c697"
Jul 30 20:10:55.047471 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:55.047432    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cc146c0-1313-4905-98d8-27bb4b43eb4d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ec1ee8-8c697\" (UID: \"4cc146c0-1313-4905-98d8-27bb4b43eb4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ec1ee8-8c697"
Jul 30 20:10:55.048101 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:55.048082    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cc146c0-1313-4905-98d8-27bb4b43eb4d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ec1ee8-8c697\" (UID: \"4cc146c0-1313-4905-98d8-27bb4b43eb4d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ec1ee8-8c697"
Jul 30 20:10:55.177240 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:55.177155    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ec1ee8-8c697"
Jul 30 20:10:55.315586 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:55.315556    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ec1ee8-8c697"]
Jul 30 20:10:55.316932 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:10:55.316895    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cc146c0_1313_4905_98d8_27bb4b43eb4d.slice/crio-60bdbc82cf14fff89a8280f9340dd394f56307a0a1dceb1b7d7bbde30d251e93 WatchSource:0}: Error finding container 60bdbc82cf14fff89a8280f9340dd394f56307a0a1dceb1b7d7bbde30d251e93: Status 404 returned error can't find the container with id 60bdbc82cf14fff89a8280f9340dd394f56307a0a1dceb1b7d7bbde30d251e93
Jul 30 20:10:55.479417 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:55.479379    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ec1ee8-8c697" event={"ID":"4cc146c0-1313-4905-98d8-27bb4b43eb4d","Type":"ContainerStarted","Data":"60bdbc82cf14fff89a8280f9340dd394f56307a0a1dceb1b7d7bbde30d251e93"}
Jul 30 20:10:56.487098 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:56.487056    2554 generic.go:356] "Generic (PLEG): container finished" podID="4cc146c0-1313-4905-98d8-27bb4b43eb4d" containerID="80ad5c31c78234811d868b17ea15cbc02c5f9c03bb0d9f8d99fe74b93a99d5cb" exitCode=0
Jul 30 20:10:56.488045 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:56.488013    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ec1ee8-8c697" event={"ID":"4cc146c0-1313-4905-98d8-27bb4b43eb4d","Type":"ContainerDied","Data":"80ad5c31c78234811d868b17ea15cbc02c5f9c03bb0d9f8d99fe74b93a99d5cb"}
Jul 30 20:10:57.638930 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:57.638905    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ec1ee8-8c697"
Jul 30 20:10:57.671565 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:57.671534    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cc146c0-1313-4905-98d8-27bb4b43eb4d-maas-api-service-ca\") pod \"4cc146c0-1313-4905-98d8-27bb4b43eb4d\" (UID: \"4cc146c0-1313-4905-98d8-27bb4b43eb4d\") "
Jul 30 20:10:57.671907 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:57.671885    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cc146c0-1313-4905-98d8-27bb4b43eb4d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4cc146c0-1313-4905-98d8-27bb4b43eb4d" (UID: "4cc146c0-1313-4905-98d8-27bb4b43eb4d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:10:57.773307 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:57.773221    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cc146c0-1313-4905-98d8-27bb4b43eb4d-maas-api-service-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:10:58.505167 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:58.505132    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ec1ee8-8c697"
Jul 30 20:10:58.505167 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:58.505151    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ec1ee8-8c697" event={"ID":"4cc146c0-1313-4905-98d8-27bb4b43eb4d","Type":"ContainerDied","Data":"60bdbc82cf14fff89a8280f9340dd394f56307a0a1dceb1b7d7bbde30d251e93"}
Jul 30 20:10:58.505376 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:10:58.505203    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="60bdbc82cf14fff89a8280f9340dd394f56307a0a1dceb1b7d7bbde30d251e93"
Jul 30 20:11:10.248188 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.248150    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk"]
Jul 30 20:11:10.248649 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.248488    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk" podUID="0ad293db-6103-4fda-aae3-198c127d55f2" containerName="maas-api" containerID="cri-o://0491c2c72e86899b713e034eba1a17a4af17d1b49c45821479a110e53c75a74d" gracePeriod=30
Jul 30 20:11:10.278674 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.278643    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:11:10.339854 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.339819    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:11:10.395447 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.395405    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:11:10.452418 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.452376    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ec1ee8-8c697"]
Jul 30 20:11:10.454179 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.454151    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:11:10.455340 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.455313    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ec1ee8-8c697"]
Jul 30 20:11:10.488883 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.488853    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cc146c0-1313-4905-98d8-27bb4b43eb4d" path="/var/lib/kubelet/pods/4cc146c0-1313-4905-98d8-27bb4b43eb4d/volumes"
Jul 30 20:11:10.515439 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.515413    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk"
Jul 30 20:11:10.583391 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.583357    2554 generic.go:356] "Generic (PLEG): container finished" podID="0ad293db-6103-4fda-aae3-198c127d55f2" containerID="0491c2c72e86899b713e034eba1a17a4af17d1b49c45821479a110e53c75a74d" exitCode=0
Jul 30 20:11:10.583579 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.583404    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk"
Jul 30 20:11:10.583579 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.583443    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk" event={"ID":"0ad293db-6103-4fda-aae3-198c127d55f2","Type":"ContainerDied","Data":"0491c2c72e86899b713e034eba1a17a4af17d1b49c45821479a110e53c75a74d"}
Jul 30 20:11:10.583579 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.583495    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk" event={"ID":"0ad293db-6103-4fda-aae3-198c127d55f2","Type":"ContainerDied","Data":"3a4f3a429633314dedd616ea11a7941db8c96d243c4e0f514545ad479253030d"}
Jul 30 20:11:10.583579 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.583518    2554 scope.go:117] "RemoveContainer" containerID="0491c2c72e86899b713e034eba1a17a4af17d1b49c45821479a110e53c75a74d"
Jul 30 20:11:10.592607 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.592588    2554 scope.go:117] "RemoveContainer" containerID="0491c2c72e86899b713e034eba1a17a4af17d1b49c45821479a110e53c75a74d"
Jul 30 20:11:10.592882 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:11:10.592860    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0491c2c72e86899b713e034eba1a17a4af17d1b49c45821479a110e53c75a74d\": container with ID starting with 0491c2c72e86899b713e034eba1a17a4af17d1b49c45821479a110e53c75a74d not found: ID does not exist" containerID="0491c2c72e86899b713e034eba1a17a4af17d1b49c45821479a110e53c75a74d"
Jul 30 20:11:10.592987 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.592894    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0491c2c72e86899b713e034eba1a17a4af17d1b49c45821479a110e53c75a74d"} err="failed to get container status \"0491c2c72e86899b713e034eba1a17a4af17d1b49c45821479a110e53c75a74d\": rpc error: code = NotFound desc = could not find container \"0491c2c72e86899b713e034eba1a17a4af17d1b49c45821479a110e53c75a74d\": container with ID starting with 0491c2c72e86899b713e034eba1a17a4af17d1b49c45821479a110e53c75a74d not found: ID does not exist"
Jul 30 20:11:10.599135 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.599114    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad293db-6103-4fda-aae3-198c127d55f2-openshift-service-ca\") pod \"0ad293db-6103-4fda-aae3-198c127d55f2\" (UID: \"0ad293db-6103-4fda-aae3-198c127d55f2\") "
Jul 30 20:11:10.599254 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.599239    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ad293db-6103-4fda-aae3-198c127d55f2-maas-api-tls\") pod \"0ad293db-6103-4fda-aae3-198c127d55f2\" (UID: \"0ad293db-6103-4fda-aae3-198c127d55f2\") "
Jul 30 20:11:10.599428 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.599413    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2c8m\" (UniqueName: \"kubernetes.io/projected/0ad293db-6103-4fda-aae3-198c127d55f2-kube-api-access-w2c8m\") pod \"0ad293db-6103-4fda-aae3-198c127d55f2\" (UID: \"0ad293db-6103-4fda-aae3-198c127d55f2\") "
Jul 30 20:11:10.599479 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.599462    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ad293db-6103-4fda-aae3-198c127d55f2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0ad293db-6103-4fda-aae3-198c127d55f2" (UID: "0ad293db-6103-4fda-aae3-198c127d55f2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:11:10.599680 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.599652    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad293db-6103-4fda-aae3-198c127d55f2-openshift-service-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:11:10.601240 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.601218    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ad293db-6103-4fda-aae3-198c127d55f2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0ad293db-6103-4fda-aae3-198c127d55f2" (UID: "0ad293db-6103-4fda-aae3-198c127d55f2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:11:10.601377 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.601360    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ad293db-6103-4fda-aae3-198c127d55f2-kube-api-access-w2c8m" (OuterVolumeSpecName: "kube-api-access-w2c8m") pod "0ad293db-6103-4fda-aae3-198c127d55f2" (UID: "0ad293db-6103-4fda-aae3-198c127d55f2"). InnerVolumeSpecName "kube-api-access-w2c8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:11:10.700368 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.700336    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2c8m\" (UniqueName: \"kubernetes.io/projected/0ad293db-6103-4fda-aae3-198c127d55f2-kube-api-access-w2c8m\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:11:10.700368 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.700366    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ad293db-6103-4fda-aae3-198c127d55f2-maas-api-tls\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:11:10.804016 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.803917    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:11:10.906624 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.906586    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk"]
Jul 30 20:11:10.909009 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:10.908974    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ec1ee8-dd5746dd8-bzkjk"]
Jul 30 20:11:12.487173 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:12.487138    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ad293db-6103-4fda-aae3-198c127d55f2" path="/var/lib/kubelet/pods/0ad293db-6103-4fda-aae3-198c127d55f2/volumes"
Jul 30 20:11:16.477900 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:16.477861    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:11:16.531962 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:16.531913    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:11:16.741113 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:16.741022    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:11:16.966287 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:16.966253    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:11:17.081333 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:17.081251    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-18459c-6mrx6"]
Jul 30 20:11:17.081666 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:17.081654    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cc146c0-1313-4905-98d8-27bb4b43eb4d" containerName="revoke-keys"
Jul 30 20:11:17.081712 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:17.081667    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cc146c0-1313-4905-98d8-27bb4b43eb4d" containerName="revoke-keys"
Jul 30 20:11:17.081712 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:17.081689    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ad293db-6103-4fda-aae3-198c127d55f2" containerName="maas-api"
Jul 30 20:11:17.081712 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:17.081696    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ad293db-6103-4fda-aae3-198c127d55f2" containerName="maas-api"
Jul 30 20:11:17.081861 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:17.081776    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cc146c0-1313-4905-98d8-27bb4b43eb4d" containerName="revoke-keys"
Jul 30 20:11:17.081861 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:17.081787    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ad293db-6103-4fda-aae3-198c127d55f2" containerName="maas-api"
Jul 30 20:11:17.085132 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:17.085112    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-18459c-6mrx6"
Jul 30 20:11:17.087412 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:17.087391    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7t7kk\""
Jul 30 20:11:17.092919 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:17.092897    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-18459c-6mrx6"]
Jul 30 20:11:17.263621 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:17.263584    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bc31d0d-6a1a-4a81-897c-9727cb9ed544-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-18459c-6mrx6\" (UID: \"0bc31d0d-6a1a-4a81-897c-9727cb9ed544\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-18459c-6mrx6"
Jul 30 20:11:17.364524 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:17.364436    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bc31d0d-6a1a-4a81-897c-9727cb9ed544-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-18459c-6mrx6\" (UID: \"0bc31d0d-6a1a-4a81-897c-9727cb9ed544\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-18459c-6mrx6"
Jul 30 20:11:17.365126 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:17.365106    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bc31d0d-6a1a-4a81-897c-9727cb9ed544-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-18459c-6mrx6\" (UID: \"0bc31d0d-6a1a-4a81-897c-9727cb9ed544\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-18459c-6mrx6"
Jul 30 20:11:17.397500 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:17.397469    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-18459c-6mrx6"
Jul 30 20:11:17.539134 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:17.539088    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-18459c-6mrx6"]
Jul 30 20:11:17.539507 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:11:17.539348    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bc31d0d_6a1a_4a81_897c_9727cb9ed544.slice/crio-026b22143d82a6f613579af0d5cdba4a6755c6f3b00e7b9eeb5badcf7aece270 WatchSource:0}: Error finding container 026b22143d82a6f613579af0d5cdba4a6755c6f3b00e7b9eeb5badcf7aece270: Status 404 returned error can't find the container with id 026b22143d82a6f613579af0d5cdba4a6755c6f3b00e7b9eeb5badcf7aece270
Jul 30 20:11:17.626601 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:17.626509    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-18459c-6mrx6" event={"ID":"0bc31d0d-6a1a-4a81-897c-9727cb9ed544","Type":"ContainerStarted","Data":"026b22143d82a6f613579af0d5cdba4a6755c6f3b00e7b9eeb5badcf7aece270"}
Jul 30 20:11:18.634777 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:18.634684    2554 generic.go:356] "Generic (PLEG): container finished" podID="0bc31d0d-6a1a-4a81-897c-9727cb9ed544" containerID="495419814c16d4d721008c10649732b634b83156d55fc65f5edab9b0f96b475f" exitCode=0
Jul 30 20:11:18.635182 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:18.634772    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-18459c-6mrx6" event={"ID":"0bc31d0d-6a1a-4a81-897c-9727cb9ed544","Type":"ContainerDied","Data":"495419814c16d4d721008c10649732b634b83156d55fc65f5edab9b0f96b475f"}
Jul 30 20:11:19.782172 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:19.782142    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-18459c-6mrx6"
Jul 30 20:11:19.888380 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:19.888346    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bc31d0d-6a1a-4a81-897c-9727cb9ed544-maas-api-service-ca\") pod \"0bc31d0d-6a1a-4a81-897c-9727cb9ed544\" (UID: \"0bc31d0d-6a1a-4a81-897c-9727cb9ed544\") "
Jul 30 20:11:19.888758 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:19.888736    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bc31d0d-6a1a-4a81-897c-9727cb9ed544-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0bc31d0d-6a1a-4a81-897c-9727cb9ed544" (UID: "0bc31d0d-6a1a-4a81-897c-9727cb9ed544"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:11:19.989717 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:19.989687    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bc31d0d-6a1a-4a81-897c-9727cb9ed544-maas-api-service-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:11:20.648691 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:20.648657    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-18459c-6mrx6"
Jul 30 20:11:20.648691 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:20.648684    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-18459c-6mrx6" event={"ID":"0bc31d0d-6a1a-4a81-897c-9727cb9ed544","Type":"ContainerDied","Data":"026b22143d82a6f613579af0d5cdba4a6755c6f3b00e7b9eeb5badcf7aece270"}
Jul 30 20:11:20.648909 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:20.648756    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="026b22143d82a6f613579af0d5cdba4a6755c6f3b00e7b9eeb5badcf7aece270"
Jul 30 20:11:32.582999 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:32.582944    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg"]
Jul 30 20:11:32.583548 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:32.583217    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg" podUID="6f4d75d8-6d9b-4c69-85ae-6951b669228f" containerName="maas-api" containerID="cri-o://bb7cd8761dce8855aab17993173a7f2105e980a096f8b69e2a515e93eec8b034" gracePeriod=30
Jul 30 20:11:32.607329 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:32.607299    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:11:32.672972 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:32.672921    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:11:32.722596 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:32.722546    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:11:32.724690 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:32.724666    2554 generic.go:356] "Generic (PLEG): container finished" podID="6f4d75d8-6d9b-4c69-85ae-6951b669228f" containerID="bb7cd8761dce8855aab17993173a7f2105e980a096f8b69e2a515e93eec8b034" exitCode=0
Jul 30 20:11:32.724807 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:32.724699    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg" event={"ID":"6f4d75d8-6d9b-4c69-85ae-6951b669228f","Type":"ContainerDied","Data":"bb7cd8761dce8855aab17993173a7f2105e980a096f8b69e2a515e93eec8b034"}
Jul 30 20:11:32.780440 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:32.780381    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-18459c-6mrx6"]
Jul 30 20:11:32.782453 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:32.782419    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:11:32.785007 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:32.784980    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-18459c-6mrx6"]
Jul 30 20:11:32.844483 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:32.844422    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg"
Jul 30 20:11:32.910645 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:32.910608    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f4d75d8-6d9b-4c69-85ae-6951b669228f-openshift-service-ca\") pod \"6f4d75d8-6d9b-4c69-85ae-6951b669228f\" (UID: \"6f4d75d8-6d9b-4c69-85ae-6951b669228f\") "
Jul 30 20:11:32.910859 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:32.910714    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-plrzm\" (UniqueName: \"kubernetes.io/projected/6f4d75d8-6d9b-4c69-85ae-6951b669228f-kube-api-access-plrzm\") pod \"6f4d75d8-6d9b-4c69-85ae-6951b669228f\" (UID: \"6f4d75d8-6d9b-4c69-85ae-6951b669228f\") "
Jul 30 20:11:32.910859 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:32.910776    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f4d75d8-6d9b-4c69-85ae-6951b669228f-maas-api-tls\") pod \"6f4d75d8-6d9b-4c69-85ae-6951b669228f\" (UID: \"6f4d75d8-6d9b-4c69-85ae-6951b669228f\") "
Jul 30 20:11:32.910993 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:32.910971    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f4d75d8-6d9b-4c69-85ae-6951b669228f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6f4d75d8-6d9b-4c69-85ae-6951b669228f" (UID: "6f4d75d8-6d9b-4c69-85ae-6951b669228f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:11:32.911123 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:32.911106    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f4d75d8-6d9b-4c69-85ae-6951b669228f-openshift-service-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:11:32.912898 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:32.912864    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f4d75d8-6d9b-4c69-85ae-6951b669228f-kube-api-access-plrzm" (OuterVolumeSpecName: "kube-api-access-plrzm") pod "6f4d75d8-6d9b-4c69-85ae-6951b669228f" (UID: "6f4d75d8-6d9b-4c69-85ae-6951b669228f"). InnerVolumeSpecName "kube-api-access-plrzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:11:32.913064 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:32.912913    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f4d75d8-6d9b-4c69-85ae-6951b669228f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6f4d75d8-6d9b-4c69-85ae-6951b669228f" (UID: "6f4d75d8-6d9b-4c69-85ae-6951b669228f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:11:33.012458 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:33.012421    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plrzm\" (UniqueName: \"kubernetes.io/projected/6f4d75d8-6d9b-4c69-85ae-6951b669228f-kube-api-access-plrzm\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:11:33.012458 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:33.012454    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f4d75d8-6d9b-4c69-85ae-6951b669228f-maas-api-tls\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:11:33.164617 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:33.164533    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:11:33.732912 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:33.732883    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg"
Jul 30 20:11:33.732912 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:33.732899    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg" event={"ID":"6f4d75d8-6d9b-4c69-85ae-6951b669228f","Type":"ContainerDied","Data":"0345bf30664a13d0213d8252c0facbaa07e6f8faec58e1132b0ab4e71fd0f6b4"}
Jul 30 20:11:33.733442 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:33.732982    2554 scope.go:117] "RemoveContainer" containerID="bb7cd8761dce8855aab17993173a7f2105e980a096f8b69e2a515e93eec8b034"
Jul 30 20:11:33.754989 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:33.754940    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg"]
Jul 30 20:11:33.758277 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:33.758254    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-18459c-6fff6cfd98-cjvsg"]
Jul 30 20:11:34.487797 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:34.487756    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bc31d0d-6a1a-4a81-897c-9727cb9ed544" path="/var/lib/kubelet/pods/0bc31d0d-6a1a-4a81-897c-9727cb9ed544/volumes"
Jul 30 20:11:34.488745 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:34.488722    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f4d75d8-6d9b-4c69-85ae-6951b669228f" path="/var/lib/kubelet/pods/6f4d75d8-6d9b-4c69-85ae-6951b669228f/volumes"
Jul 30 20:11:39.026735 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:39.026702    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:11:39.126912 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:39.126875    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:11:39.352684 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:39.352585    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:11:39.550991 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:39.550946    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:11:39.879184 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:39.879142    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:11:40.192039 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.192005    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"]
Jul 30 20:11:40.192411 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.192403    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f4d75d8-6d9b-4c69-85ae-6951b669228f" containerName="maas-api"
Jul 30 20:11:40.192453 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.192414    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f4d75d8-6d9b-4c69-85ae-6951b669228f" containerName="maas-api"
Jul 30 20:11:40.192453 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.192429    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bc31d0d-6a1a-4a81-897c-9727cb9ed544" containerName="revoke-keys"
Jul 30 20:11:40.192453 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.192436    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bc31d0d-6a1a-4a81-897c-9727cb9ed544" containerName="revoke-keys"
Jul 30 20:11:40.192557 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.192508    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bc31d0d-6a1a-4a81-897c-9727cb9ed544" containerName="revoke-keys"
Jul 30 20:11:40.192557 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.192519    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f4d75d8-6d9b-4c69-85ae-6951b669228f" containerName="maas-api"
Jul 30 20:11:40.196835 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.196819    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.199384 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.199361    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-ecda00-openshift-default-dockercfg-lj5gf\""
Jul 30 20:11:40.209238 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.209212    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"]
Jul 30 20:11:40.213985 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.213939    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:11:40.287685 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.287636    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istio-data\") pod \"e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") " pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.287899 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.287708    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istio-envoy\") pod \"e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") " pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.287899 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.287744    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-workload-socket\") pod \"e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") " pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.287899 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.287775    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istio-token\") pod \"e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") " pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.287899 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.287806    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istiod-ca-cert\") pod \"e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") " pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.287899 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.287850    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2k2qw\" (UniqueName: \"kubernetes.io/projected/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-kube-api-access-2k2qw\") pod \"e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") " pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.287899 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.287888    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-credential-socket\") pod \"e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") " pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.288283 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.287923    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-workload-certs\") pod \"e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") " pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.288283 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.287994    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istio-podinfo\") pod \"e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") " pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.388490 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.388454    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istiod-ca-cert\") pod \"e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") " pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.388490 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.388506    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2k2qw\" (UniqueName: \"kubernetes.io/projected/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-kube-api-access-2k2qw\") pod \"e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") " pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.388751 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.388647    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-credential-socket\") pod \"e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") " pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.388817 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.388776    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-workload-certs\") pod \"e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") " pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.388872 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.388841    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istio-podinfo\") pod \"e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") " pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.388989 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.388970    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istio-data\") pod \"e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") " pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.389059 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.389009    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-credential-socket\") pod \"e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") " pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.389059 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.389010    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istio-envoy\") pod \"e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") " pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.389169 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.389081    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-workload-socket\") pod \"e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") " pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.389169 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.389106    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-workload-certs\") pod \"e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") " pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.389169 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.389114    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istio-token\") pod \"e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") " pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.389323 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.389204    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istiod-ca-cert\") pod \"e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") " pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.389374 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.389341    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istio-data\") pod \"e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") " pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.389418 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.389396    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-workload-socket\") pod \"e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") " pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.391298 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.391277    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istio-envoy\") pod \"e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") " pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.391383 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.391355    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istio-podinfo\") pod \"e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") " pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.395993 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.395943    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2k2qw\" (UniqueName: \"kubernetes.io/projected/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-kube-api-access-2k2qw\") pod \"e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") " pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.396196 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.396180    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istio-token\") pod \"e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") " pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.483141 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.483099    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:11:40.508125 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.508096    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:40.656098 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.656064    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"]
Jul 30 20:11:40.657186 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:11:40.657155    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c9ed2ee_aae2_41dc_b31b_3207dab9e52e.slice/crio-3e2e515d69cb0f2a164f9d674af92a5958bdf1b4f535bd30fee8c8b8c6d7c8ce WatchSource:0}: Error finding container 3e2e515d69cb0f2a164f9d674af92a5958bdf1b4f535bd30fee8c8b8c6d7c8ce: Status 404 returned error can't find the container with id 3e2e515d69cb0f2a164f9d674af92a5958bdf1b4f535bd30fee8c8b8c6d7c8ce
Jul 30 20:11:40.659478 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.659442    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 20:11:40.659574 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.659526    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 20:11:40.659637 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.659572    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 20:11:40.778560 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.778456    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc" event={"ID":"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e","Type":"ContainerStarted","Data":"f569cf59185260ac3ad0023d2c4fa514b7fce9f2f3ca7d969a545eb82681ad57"}
Jul 30 20:11:40.778560 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.778528    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc" event={"ID":"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e","Type":"ContainerStarted","Data":"3e2e515d69cb0f2a164f9d674af92a5958bdf1b4f535bd30fee8c8b8c6d7c8ce"}
Jul 30 20:11:40.804035 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:40.803974    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc" podStartSLOduration=0.803938393 podStartE2EDuration="803.938393ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:11:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:11:40.801284957 +0000 UTC m=+2828.765866053" watchObservedRunningTime="2026-07-30 20:11:40.803938393 +0000 UTC m=+2828.768519475"
Jul 30 20:11:41.508408 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:41.508369    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:41.514697 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:41.514667    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:41.784685 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:41.784589    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:41.785502 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:41.785482    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:42.751117 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:42.751083    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:11:42.841335 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:42.841292    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:11:43.845562 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:43.845531    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n"]
Jul 30 20:11:43.849148 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:43.849129    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n"
Jul 30 20:11:43.851664 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:43.851644    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-ecda00-serving-cert\""
Jul 30 20:11:43.859981 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:43.859937    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n"]
Jul 30 20:11:43.874807 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:43.874774    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:11:43.922903 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:43.922867    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssgwr\" (UniqueName: \"kubernetes.io/projected/811a3018-aa91-4f66-9dfd-2eb7baf8c3d5-kube-api-access-ssgwr\") pod \"maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n\" (UID: \"811a3018-aa91-4f66-9dfd-2eb7baf8c3d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n"
Jul 30 20:11:43.923106 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:43.922918    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/811a3018-aa91-4f66-9dfd-2eb7baf8c3d5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n\" (UID: \"811a3018-aa91-4f66-9dfd-2eb7baf8c3d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n"
Jul 30 20:11:43.923106 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:43.923005    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/811a3018-aa91-4f66-9dfd-2eb7baf8c3d5-maas-api-tls\") pod \"maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n\" (UID: \"811a3018-aa91-4f66-9dfd-2eb7baf8c3d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n"
Jul 30 20:11:44.024388 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:44.024351    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/811a3018-aa91-4f66-9dfd-2eb7baf8c3d5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n\" (UID: \"811a3018-aa91-4f66-9dfd-2eb7baf8c3d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n"
Jul 30 20:11:44.024579 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:44.024402    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/811a3018-aa91-4f66-9dfd-2eb7baf8c3d5-maas-api-tls\") pod \"maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n\" (UID: \"811a3018-aa91-4f66-9dfd-2eb7baf8c3d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n"
Jul 30 20:11:44.024579 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:44.024501    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssgwr\" (UniqueName: \"kubernetes.io/projected/811a3018-aa91-4f66-9dfd-2eb7baf8c3d5-kube-api-access-ssgwr\") pod \"maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n\" (UID: \"811a3018-aa91-4f66-9dfd-2eb7baf8c3d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n"
Jul 30 20:11:44.024676 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:11:44.024600    2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-serving-cert: secret "maas-api-e2e-shared-a-ecda00-serving-cert" not found
Jul 30 20:11:44.024716 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:11:44.024684    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/811a3018-aa91-4f66-9dfd-2eb7baf8c3d5-maas-api-tls podName:811a3018-aa91-4f66-9dfd-2eb7baf8c3d5 nodeName:}" failed. No retries permitted until 2026-07-30 20:11:44.524663477 +0000 UTC m=+2832.489244539 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/811a3018-aa91-4f66-9dfd-2eb7baf8c3d5-maas-api-tls") pod "maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n" (UID: "811a3018-aa91-4f66-9dfd-2eb7baf8c3d5") : secret "maas-api-e2e-shared-a-ecda00-serving-cert" not found
Jul 30 20:11:44.025031 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:44.025014    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/811a3018-aa91-4f66-9dfd-2eb7baf8c3d5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n\" (UID: \"811a3018-aa91-4f66-9dfd-2eb7baf8c3d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n"
Jul 30 20:11:44.038363 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:44.038336    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssgwr\" (UniqueName: \"kubernetes.io/projected/811a3018-aa91-4f66-9dfd-2eb7baf8c3d5-kube-api-access-ssgwr\") pod \"maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n\" (UID: \"811a3018-aa91-4f66-9dfd-2eb7baf8c3d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n"
Jul 30 20:11:44.358575 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:44.358540    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:11:44.456160 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:44.456107    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:11:44.528239 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:44.528209    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/811a3018-aa91-4f66-9dfd-2eb7baf8c3d5-maas-api-tls\") pod \"maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n\" (UID: \"811a3018-aa91-4f66-9dfd-2eb7baf8c3d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n"
Jul 30 20:11:44.528426 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:11:44.528370    2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-serving-cert: secret "maas-api-e2e-shared-a-ecda00-serving-cert" not found
Jul 30 20:11:44.528497 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:11:44.528443    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/811a3018-aa91-4f66-9dfd-2eb7baf8c3d5-maas-api-tls podName:811a3018-aa91-4f66-9dfd-2eb7baf8c3d5 nodeName:}" failed. No retries permitted until 2026-07-30 20:11:45.528421964 +0000 UTC m=+2833.493003024 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/811a3018-aa91-4f66-9dfd-2eb7baf8c3d5-maas-api-tls") pod "maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n" (UID: "811a3018-aa91-4f66-9dfd-2eb7baf8c3d5") : secret "maas-api-e2e-shared-a-ecda00-serving-cert" not found
Jul 30 20:11:44.825778 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:44.825743    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:11:45.538644 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:45.538607    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/811a3018-aa91-4f66-9dfd-2eb7baf8c3d5-maas-api-tls\") pod \"maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n\" (UID: \"811a3018-aa91-4f66-9dfd-2eb7baf8c3d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n"
Jul 30 20:11:45.541084 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:45.541061    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/811a3018-aa91-4f66-9dfd-2eb7baf8c3d5-maas-api-tls\") pod \"maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n\" (UID: \"811a3018-aa91-4f66-9dfd-2eb7baf8c3d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n"
Jul 30 20:11:45.662203 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:45.662167    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n"
Jul 30 20:11:45.805632 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:45.805596    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n"]
Jul 30 20:11:45.806249 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:11:45.806218    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod811a3018_aa91_4f66_9dfd_2eb7baf8c3d5.slice/crio-216e4f0683e6b558a7162798093d7bb9f41d46cd8ca551a45fa0f9d05c1d1d1d WatchSource:0}: Error finding container 216e4f0683e6b558a7162798093d7bb9f41d46cd8ca551a45fa0f9d05c1d1d1d: Status 404 returned error can't find the container with id 216e4f0683e6b558a7162798093d7bb9f41d46cd8ca551a45fa0f9d05c1d1d1d
Jul 30 20:11:46.816992 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:46.816944    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n" event={"ID":"811a3018-aa91-4f66-9dfd-2eb7baf8c3d5","Type":"ContainerStarted","Data":"1f8aa6787ebfd22126d9ef50c6041891362840c5f5f3aa0d719d1b8ce1e031c0"}
Jul 30 20:11:46.817544 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:46.817016    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n" event={"ID":"811a3018-aa91-4f66-9dfd-2eb7baf8c3d5","Type":"ContainerStarted","Data":"216e4f0683e6b558a7162798093d7bb9f41d46cd8ca551a45fa0f9d05c1d1d1d"}
Jul 30 20:11:46.817544 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:46.817165    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n"
Jul 30 20:11:46.836600 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:46.836551    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n" podStartSLOduration=3.456941734 podStartE2EDuration="3.836537088s" totalImagesPullingTime="379.595354ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:11:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:11:46.833124376 +0000 UTC m=+2834.797705455" watchObservedRunningTime="2026-07-30 20:11:46.836537088 +0000 UTC m=+2834.801118225"
Jul 30 20:11:52.836324 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:52.836287    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n"
Jul 30 20:11:56.786973 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:56.786924    2554 patch_prober.go:28] interesting pod/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Jul 30 20:11:56.787472 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:56.787006    2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc" podUID="2c9ed2ee-aae2-41dc-b31b-3207dab9e52e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Jul 30 20:11:56.890212 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:56.890176    2554 generic.go:356] "Generic (PLEG): container finished" podID="2c9ed2ee-aae2-41dc-b31b-3207dab9e52e" containerID="f569cf59185260ac3ad0023d2c4fa514b7fce9f2f3ca7d969a545eb82681ad57" exitCode=0
Jul 30 20:11:56.890386 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:56.890252    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc" event={"ID":"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e","Type":"ContainerDied","Data":"f569cf59185260ac3ad0023d2c4fa514b7fce9f2f3ca7d969a545eb82681ad57"}
Jul 30 20:11:56.890664 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:56.890650    2554 scope.go:117] "RemoveContainer" containerID="f569cf59185260ac3ad0023d2c4fa514b7fce9f2f3ca7d969a545eb82681ad57"
Jul 30 20:11:56.891822 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:56.891794    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 20:11:56.891881 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:56.891871    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 20:11:56.891922 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:56.891909    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 20:11:57.898107 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:57.898063    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc" event={"ID":"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e","Type":"ContainerStarted","Data":"046f66345afe96ef9f0c7e35e2480e9a51ab02732bb85757a764d8043aafa00b"}
Jul 30 20:11:58.509369 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:58.509334    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:11:58.509587 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:58.509385    2554 patch_prober.go:28] interesting pod/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Jul 30 20:11:58.509587 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:58.509441    2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc" podUID="2c9ed2ee-aae2-41dc-b31b-3207dab9e52e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Jul 30 20:11:59.519389 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:59.519352    2554 patch_prober.go:28] interesting pod/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Jul 30 20:11:59.520006 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:11:59.519972    2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc" podUID="2c9ed2ee-aae2-41dc-b31b-3207dab9e52e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Jul 30 20:12:00.509415 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:12:00.509369    2554 patch_prober.go:28] interesting pod/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Jul 30 20:12:00.509628 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:12:00.509478    2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc" podUID="2c9ed2ee-aae2-41dc-b31b-3207dab9e52e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Jul 30 20:12:01.508683 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:12:01.508636    2554 patch_prober.go:28] interesting pod/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Jul 30 20:12:01.509214 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:12:01.508715    2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc" podUID="2c9ed2ee-aae2-41dc-b31b-3207dab9e52e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Jul 30 20:12:02.512922 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:12:02.512881    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:12:02.513402 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:12:02.513247    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:12:02.513811 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:12:02.513797    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:12:07.725981 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:12:07.725918    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:12:08.964202 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:12:08.964167    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:12:11.412637 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:12:11.412604    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:12:11.510939 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:12:11.510892    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:12:12.548411 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:12:12.548365    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:12:12.850184 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:12:12.850096    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:12:13.029238 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:12:13.029195    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:12:13.134414 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:12:13.134319    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:12:36.401530 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:12:36.401496    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:12:46.783501 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:12:46.783462    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:13:22.175298 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:22.175260    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"]
Jul 30 20:13:22.178873 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:22.178856    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"
Jul 30 20:13:22.181445 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:22.181423    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a769e5\"/\"openshift-service-ca.crt\""
Jul 30 20:13:22.181553 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:22.181428    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a769e5\"/\"kube-root-ca.crt\""
Jul 30 20:13:22.182529 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:22.182506    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a769e5\"/\"default-dockercfg-gprzj\""
Jul 30 20:13:22.182638 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:22.182535    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a769e5\"/\"auth-test-model-a769e5-kserve-self-signed-certs\""
Jul 30 20:13:22.188304 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:22.188277    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"]
Jul 30 20:13:22.267600 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:22.267560    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:13:22.284107 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:22.284081    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8e913fa-d34e-4134-90f0-7a08d632ec13-tls-certs\") pod \"auth-test-model-a769e5-kserve-7bb8894895-j6fkc\" (UID: \"a8e913fa-d34e-4134-90f0-7a08d632ec13\") " pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"
Jul 30 20:13:22.284229 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:22.284118    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8e913fa-d34e-4134-90f0-7a08d632ec13-dshm\") pod \"auth-test-model-a769e5-kserve-7bb8894895-j6fkc\" (UID: \"a8e913fa-d34e-4134-90f0-7a08d632ec13\") " pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"
Jul 30 20:13:22.284229 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:22.284140    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8e913fa-d34e-4134-90f0-7a08d632ec13-home\") pod \"auth-test-model-a769e5-kserve-7bb8894895-j6fkc\" (UID: \"a8e913fa-d34e-4134-90f0-7a08d632ec13\") " pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"
Jul 30 20:13:22.284309 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:22.284264    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8e913fa-d34e-4134-90f0-7a08d632ec13-tmp-dir\") pod \"auth-test-model-a769e5-kserve-7bb8894895-j6fkc\" (UID: \"a8e913fa-d34e-4134-90f0-7a08d632ec13\") " pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"
Jul 30 20:13:22.284309 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:22.284298    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8e913fa-d34e-4134-90f0-7a08d632ec13-model-cache\") pod \"auth-test-model-a769e5-kserve-7bb8894895-j6fkc\" (UID: \"a8e913fa-d34e-4134-90f0-7a08d632ec13\") " pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"
Jul 30 20:13:22.284378 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:22.284318    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pj72\" (UniqueName: \"kubernetes.io/projected/a8e913fa-d34e-4134-90f0-7a08d632ec13-kube-api-access-8pj72\") pod \"auth-test-model-a769e5-kserve-7bb8894895-j6fkc\" (UID: \"a8e913fa-d34e-4134-90f0-7a08d632ec13\") " pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"
Jul 30 20:13:22.385811 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:22.385765    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8e913fa-d34e-4134-90f0-7a08d632ec13-tmp-dir\") pod \"auth-test-model-a769e5-kserve-7bb8894895-j6fkc\" (UID: \"a8e913fa-d34e-4134-90f0-7a08d632ec13\") " pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"
Jul 30 20:13:22.386033 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:22.385825    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8e913fa-d34e-4134-90f0-7a08d632ec13-model-cache\") pod \"auth-test-model-a769e5-kserve-7bb8894895-j6fkc\" (UID: \"a8e913fa-d34e-4134-90f0-7a08d632ec13\") " pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"
Jul 30 20:13:22.386033 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:22.385855    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pj72\" (UniqueName: \"kubernetes.io/projected/a8e913fa-d34e-4134-90f0-7a08d632ec13-kube-api-access-8pj72\") pod \"auth-test-model-a769e5-kserve-7bb8894895-j6fkc\" (UID: \"a8e913fa-d34e-4134-90f0-7a08d632ec13\") " pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"
Jul 30 20:13:22.386033 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:22.385886    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8e913fa-d34e-4134-90f0-7a08d632ec13-tls-certs\") pod \"auth-test-model-a769e5-kserve-7bb8894895-j6fkc\" (UID: \"a8e913fa-d34e-4134-90f0-7a08d632ec13\") " pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"
Jul 30 20:13:22.386033 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:22.385931    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8e913fa-d34e-4134-90f0-7a08d632ec13-dshm\") pod \"auth-test-model-a769e5-kserve-7bb8894895-j6fkc\" (UID: \"a8e913fa-d34e-4134-90f0-7a08d632ec13\") " pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"
Jul 30 20:13:22.386281 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:22.386098    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8e913fa-d34e-4134-90f0-7a08d632ec13-home\") pod \"auth-test-model-a769e5-kserve-7bb8894895-j6fkc\" (UID: \"a8e913fa-d34e-4134-90f0-7a08d632ec13\") " pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"
Jul 30 20:13:22.386281 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:22.386218    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8e913fa-d34e-4134-90f0-7a08d632ec13-tmp-dir\") pod \"auth-test-model-a769e5-kserve-7bb8894895-j6fkc\" (UID: \"a8e913fa-d34e-4134-90f0-7a08d632ec13\") " pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"
Jul 30 20:13:22.386397 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:22.386287    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8e913fa-d34e-4134-90f0-7a08d632ec13-model-cache\") pod \"auth-test-model-a769e5-kserve-7bb8894895-j6fkc\" (UID: \"a8e913fa-d34e-4134-90f0-7a08d632ec13\") " pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"
Jul 30 20:13:22.386480 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:22.386462    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8e913fa-d34e-4134-90f0-7a08d632ec13-home\") pod \"auth-test-model-a769e5-kserve-7bb8894895-j6fkc\" (UID: \"a8e913fa-d34e-4134-90f0-7a08d632ec13\") " pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"
Jul 30 20:13:22.388213 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:22.388193    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8e913fa-d34e-4134-90f0-7a08d632ec13-dshm\") pod \"auth-test-model-a769e5-kserve-7bb8894895-j6fkc\" (UID: \"a8e913fa-d34e-4134-90f0-7a08d632ec13\") " pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"
Jul 30 20:13:22.388480 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:22.388464    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8e913fa-d34e-4134-90f0-7a08d632ec13-tls-certs\") pod \"auth-test-model-a769e5-kserve-7bb8894895-j6fkc\" (UID: \"a8e913fa-d34e-4134-90f0-7a08d632ec13\") " pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"
Jul 30 20:13:22.394790 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:22.394766    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pj72\" (UniqueName: \"kubernetes.io/projected/a8e913fa-d34e-4134-90f0-7a08d632ec13-kube-api-access-8pj72\") pod \"auth-test-model-a769e5-kserve-7bb8894895-j6fkc\" (UID: \"a8e913fa-d34e-4134-90f0-7a08d632ec13\") " pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"
Jul 30 20:13:22.489797 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:22.489752    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"
Jul 30 20:13:22.631534 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:22.631503    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"]
Jul 30 20:13:22.632736 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:13:22.632708    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda8e913fa_d34e_4134_90f0_7a08d632ec13.slice/crio-1f08a79db638e4b74b10e9180b8a226dd25b72d99cd2a4a9a5034f80ede6dc00 WatchSource:0}: Error finding container 1f08a79db638e4b74b10e9180b8a226dd25b72d99cd2a4a9a5034f80ede6dc00: Status 404 returned error can't find the container with id 1f08a79db638e4b74b10e9180b8a226dd25b72d99cd2a4a9a5034f80ede6dc00
Jul 30 20:13:23.413625 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:23.413587    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc" event={"ID":"a8e913fa-d34e-4134-90f0-7a08d632ec13","Type":"ContainerStarted","Data":"3dd30580ad0b61c742966a6cb78aea3bd7233e2e4f7239e3b87f509aaf37f42c"}
Jul 30 20:13:23.414102 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:23.413633    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc" event={"ID":"a8e913fa-d34e-4134-90f0-7a08d632ec13","Type":"ContainerStarted","Data":"1f08a79db638e4b74b10e9180b8a226dd25b72d99cd2a4a9a5034f80ede6dc00"}
Jul 30 20:13:23.431107 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:23.431059    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc" podStartSLOduration=1.431044355 podStartE2EDuration="1.431044355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:13:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:13:23.429348938 +0000 UTC m=+2931.393930020" watchObservedRunningTime="2026-07-30 20:13:23.431044355 +0000 UTC m=+2931.395625436"
Jul 30 20:13:32.490107 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:32.490075    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"
Jul 30 20:13:32.490549 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:32.490266    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"
Jul 30 20:13:32.502512 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:32.502489    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"
Jul 30 20:13:33.486648 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:33.486614    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"
Jul 30 20:13:53.556864 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:53.556820    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:13:53.615903 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:13:53.615866    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:14:32.791072 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:14:32.790989    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/1.log"
Jul 30 20:14:32.795786 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:14:32.795766    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 20:14:32.801719 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:14:32.801698    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/1.log"
Jul 30 20:14:32.806019 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:14:32.806003    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 20:14:56.149025 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:14:56.148988    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:15:00.142853 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:00.142813    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a769e5-29757375-g74gj"]
Jul 30 20:15:00.147205 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:00.147175    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ecda00-29757375-ln47x"]
Jul 30 20:15:00.147322 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:00.147282    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a769e5-29757375-g74gj"
Jul 30 20:15:00.149904 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:00.149881    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7t7kk\""
Jul 30 20:15:00.151017 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:00.151002    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757375-bfgnl"]
Jul 30 20:15:00.151127 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:00.151110    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ecda00-29757375-ln47x"
Jul 30 20:15:00.154405 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:00.154383    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757375-bfgnl"
Jul 30 20:15:00.155574 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:00.155298    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a769e5-29757375-g74gj"]
Jul 30 20:15:00.158744 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:00.158719    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757375-bfgnl"]
Jul 30 20:15:00.160923 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:00.160905    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a769e5-29757375-g74gj"
Jul 30 20:15:00.171536 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:00.171506    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ecda00-29757375-ln47x"]
Jul 30 20:15:00.171860 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:00.171837    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ecda00-29757375-ln47x"
Jul 30 20:15:00.204407 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:00.204382    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757375-bfgnl"
Jul 30 20:15:00.339475 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:00.339443    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a769e5-29757375-g74gj"]
Jul 30 20:15:00.341599 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:15:00.341561    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdbcb8ce_e98a_4894_9fa4_3febd0fbef54.slice/crio-21bd8da7e4eadbc5ca6d703d37a7e5631ff237d07e6c633ffc896d2d0b0718b5 WatchSource:0}: Error finding container 21bd8da7e4eadbc5ca6d703d37a7e5631ff237d07e6c633ffc896d2d0b0718b5: Status 404 returned error can't find the container with id 21bd8da7e4eadbc5ca6d703d37a7e5631ff237d07e6c633ffc896d2d0b0718b5
Jul 30 20:15:00.362590 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:00.362497    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ecda00-29757375-ln47x"]
Jul 30 20:15:00.391390 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:00.391363    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29757375-bfgnl"]
Jul 30 20:15:00.391787 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:15:00.391752    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod446a5558_e343_48f5_bb07_a0a8e1dd9cf2.slice/crio-00f109591b2a18d6f09dd542dc614e7c0dbc8b36799c7e6d26f7341e769564d9 WatchSource:0}: Error finding container 00f109591b2a18d6f09dd542dc614e7c0dbc8b36799c7e6d26f7341e769564d9: Status 404 returned error can't find the container with id 00f109591b2a18d6f09dd542dc614e7c0dbc8b36799c7e6d26f7341e769564d9
Jul 30 20:15:00.992894 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:00.992794    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a769e5-29757375-g74gj" event={"ID":"bdbcb8ce-e98a-4894-9fa4-3febd0fbef54","Type":"ContainerStarted","Data":"e8229d0979cc4758bda77355ad3192076de3bd0f0ea85bb8a1c60f07fec656b7"}
Jul 30 20:15:00.992894 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:00.992879    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a769e5-29757375-g74gj" event={"ID":"bdbcb8ce-e98a-4894-9fa4-3febd0fbef54","Type":"ContainerStarted","Data":"21bd8da7e4eadbc5ca6d703d37a7e5631ff237d07e6c633ffc896d2d0b0718b5"}
Jul 30 20:15:00.994436 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:00.994410    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757375-bfgnl" event={"ID":"446a5558-e343-48f5-bb07-a0a8e1dd9cf2","Type":"ContainerStarted","Data":"937f5d2ff1b59a2923b09d44b6747b6dda0d37487b64d8d167f4ad99edf966c3"}
Jul 30 20:15:00.994568 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:00.994458    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757375-bfgnl" event={"ID":"446a5558-e343-48f5-bb07-a0a8e1dd9cf2","Type":"ContainerStarted","Data":"00f109591b2a18d6f09dd542dc614e7c0dbc8b36799c7e6d26f7341e769564d9"}
Jul 30 20:15:00.995829 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:00.995806    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ecda00-29757375-ln47x" event={"ID":"c2061f5b-ff8d-4473-9f9f-eaac8f123f5b","Type":"ContainerStarted","Data":"40c9eb1da3fb2c0ba5c2272439eb61e3ffc8e70d45fffab6708463e145760244"}
Jul 30 20:15:00.995944 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:00.995842    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ecda00-29757375-ln47x" event={"ID":"c2061f5b-ff8d-4473-9f9f-eaac8f123f5b","Type":"ContainerStarted","Data":"07271764473e48d5dec621d52fd7762f5aa8e69f8ed026b2700fe067fbeca1e8"}
Jul 30 20:15:01.012205 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:01.012149    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a769e5-29757375-g74gj" podStartSLOduration=1.012133117 podStartE2EDuration="1.012133117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:15:01.011338423 +0000 UTC m=+3028.975919505" watchObservedRunningTime="2026-07-30 20:15:01.012133117 +0000 UTC m=+3028.976714197"
Jul 30 20:15:01.029516 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:01.029460    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757375-bfgnl" podStartSLOduration=1.029442677 podStartE2EDuration="1.029442677s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:15:01.028449323 +0000 UTC m=+3028.993030402" watchObservedRunningTime="2026-07-30 20:15:01.029442677 +0000 UTC m=+3028.994023757"
Jul 30 20:15:01.047567 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:01.047511    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ecda00-29757375-ln47x" podStartSLOduration=1.047494911 podStartE2EDuration="1.047494911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:15:01.046556939 +0000 UTC m=+3029.011138020" watchObservedRunningTime="2026-07-30 20:15:01.047494911 +0000 UTC m=+3029.012075991"
Jul 30 20:15:02.003925 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:02.003889    2554 generic.go:356] "Generic (PLEG): container finished" podID="446a5558-e343-48f5-bb07-a0a8e1dd9cf2" containerID="937f5d2ff1b59a2923b09d44b6747b6dda0d37487b64d8d167f4ad99edf966c3" exitCode=0
Jul 30 20:15:02.004349 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:02.003940    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757375-bfgnl" event={"ID":"446a5558-e343-48f5-bb07-a0a8e1dd9cf2","Type":"ContainerDied","Data":"937f5d2ff1b59a2923b09d44b6747b6dda0d37487b64d8d167f4ad99edf966c3"}
Jul 30 20:15:02.005456 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:02.005433    2554 generic.go:356] "Generic (PLEG): container finished" podID="c2061f5b-ff8d-4473-9f9f-eaac8f123f5b" containerID="40c9eb1da3fb2c0ba5c2272439eb61e3ffc8e70d45fffab6708463e145760244" exitCode=0
Jul 30 20:15:02.005582 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:02.005508    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ecda00-29757375-ln47x" event={"ID":"c2061f5b-ff8d-4473-9f9f-eaac8f123f5b","Type":"ContainerDied","Data":"40c9eb1da3fb2c0ba5c2272439eb61e3ffc8e70d45fffab6708463e145760244"}
Jul 30 20:15:02.006914 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:02.006895    2554 generic.go:356] "Generic (PLEG): container finished" podID="bdbcb8ce-e98a-4894-9fa4-3febd0fbef54" containerID="e8229d0979cc4758bda77355ad3192076de3bd0f0ea85bb8a1c60f07fec656b7" exitCode=0
Jul 30 20:15:02.007033 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:02.006937    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a769e5-29757375-g74gj" event={"ID":"bdbcb8ce-e98a-4894-9fa4-3febd0fbef54","Type":"ContainerDied","Data":"e8229d0979cc4758bda77355ad3192076de3bd0f0ea85bb8a1c60f07fec656b7"}
Jul 30 20:15:03.175298 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:03.175272    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757375-bfgnl"
Jul 30 20:15:03.179692 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:03.179659    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a769e5-29757375-g74gj"
Jul 30 20:15:03.206168 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:03.206140    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ecda00-29757375-ln47x"
Jul 30 20:15:04.020265 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:04.020235    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a769e5-29757375-g74gj"
Jul 30 20:15:04.020453 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:04.020277    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a769e5-29757375-g74gj" event={"ID":"bdbcb8ce-e98a-4894-9fa4-3febd0fbef54","Type":"ContainerDied","Data":"21bd8da7e4eadbc5ca6d703d37a7e5631ff237d07e6c633ffc896d2d0b0718b5"}
Jul 30 20:15:04.020453 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:04.020321    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="21bd8da7e4eadbc5ca6d703d37a7e5631ff237d07e6c633ffc896d2d0b0718b5"
Jul 30 20:15:04.021657 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:04.021635    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757375-bfgnl"
Jul 30 20:15:04.021657 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:04.021648    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29757375-bfgnl" event={"ID":"446a5558-e343-48f5-bb07-a0a8e1dd9cf2","Type":"ContainerDied","Data":"00f109591b2a18d6f09dd542dc614e7c0dbc8b36799c7e6d26f7341e769564d9"}
Jul 30 20:15:04.021821 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:04.021686    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="00f109591b2a18d6f09dd542dc614e7c0dbc8b36799c7e6d26f7341e769564d9"
Jul 30 20:15:04.022854 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:04.022832    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ecda00-29757375-ln47x" event={"ID":"c2061f5b-ff8d-4473-9f9f-eaac8f123f5b","Type":"ContainerDied","Data":"07271764473e48d5dec621d52fd7762f5aa8e69f8ed026b2700fe067fbeca1e8"}
Jul 30 20:15:04.022948 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:04.022870    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="07271764473e48d5dec621d52fd7762f5aa8e69f8ed026b2700fe067fbeca1e8"
Jul 30 20:15:04.022948 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:04.022839    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ecda00-29757375-ln47x"
Jul 30 20:15:32.985202 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:15:32.985124    2554 scope.go:117] "RemoveContainer" containerID="ca575139a28c49b33429055487c4707e0ff965ce580a3c27847255b01ca88db1"
Jul 30 20:16:17.434210 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:17.434170    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-z6kzz"]
Jul 30 20:16:17.434710 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:17.434584    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="446a5558-e343-48f5-bb07-a0a8e1dd9cf2" containerName="cleanup"
Jul 30 20:16:17.434710 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:17.434595    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="446a5558-e343-48f5-bb07-a0a8e1dd9cf2" containerName="cleanup"
Jul 30 20:16:17.434710 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:17.434617    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2061f5b-ff8d-4473-9f9f-eaac8f123f5b" containerName="cleanup"
Jul 30 20:16:17.434710 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:17.434622    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2061f5b-ff8d-4473-9f9f-eaac8f123f5b" containerName="cleanup"
Jul 30 20:16:17.434710 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:17.434631    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdbcb8ce-e98a-4894-9fa4-3febd0fbef54" containerName="cleanup"
Jul 30 20:16:17.434710 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:17.434637    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdbcb8ce-e98a-4894-9fa4-3febd0fbef54" containerName="cleanup"
Jul 30 20:16:17.434710 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:17.434696    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="446a5558-e343-48f5-bb07-a0a8e1dd9cf2" containerName="cleanup"
Jul 30 20:16:17.434710 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:17.434709    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdbcb8ce-e98a-4894-9fa4-3febd0fbef54" containerName="cleanup"
Jul 30 20:16:17.434710 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:17.434715    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2061f5b-ff8d-4473-9f9f-eaac8f123f5b" containerName="cleanup"
Jul 30 20:16:17.437823 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:17.437801    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-z6kzz"
Jul 30 20:16:17.447386 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:17.447358    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-z6kzz"]
Jul 30 20:16:17.480005 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:17.479972    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zjcl\" (UniqueName: \"kubernetes.io/projected/a7637a53-8564-4cf7-9331-bc0fc216b4be-kube-api-access-2zjcl\") pod \"kuadrant-operator-catalog-z6kzz\" (UID: \"a7637a53-8564-4cf7-9331-bc0fc216b4be\") " pod="kuadrant-system/kuadrant-operator-catalog-z6kzz"
Jul 30 20:16:17.581417 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:17.581377    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zjcl\" (UniqueName: \"kubernetes.io/projected/a7637a53-8564-4cf7-9331-bc0fc216b4be-kube-api-access-2zjcl\") pod \"kuadrant-operator-catalog-z6kzz\" (UID: \"a7637a53-8564-4cf7-9331-bc0fc216b4be\") " pod="kuadrant-system/kuadrant-operator-catalog-z6kzz"
Jul 30 20:16:17.590622 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:17.590587    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zjcl\" (UniqueName: \"kubernetes.io/projected/a7637a53-8564-4cf7-9331-bc0fc216b4be-kube-api-access-2zjcl\") pod \"kuadrant-operator-catalog-z6kzz\" (UID: \"a7637a53-8564-4cf7-9331-bc0fc216b4be\") " pod="kuadrant-system/kuadrant-operator-catalog-z6kzz"
Jul 30 20:16:17.749756 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:17.749724    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-z6kzz"
Jul 30 20:16:17.894417 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:17.894384    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-z6kzz"]
Jul 30 20:16:17.911166 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:17.911132    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 20:16:18.488259 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:18.488219    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-z6kzz" event={"ID":"a7637a53-8564-4cf7-9331-bc0fc216b4be","Type":"ContainerStarted","Data":"a994b093beea1fec5edf6c914eec6443064b176a3248f0d9067eda328f94b738"}
Jul 30 20:16:18.488647 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:18.488276    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-z6kzz" event={"ID":"a7637a53-8564-4cf7-9331-bc0fc216b4be","Type":"ContainerStarted","Data":"ba2f01cee309fe0fa1f8d8a09eb002d33eb8d79781d8c2ac94f559669d3c8b2b"}
Jul 30 20:16:18.505948 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:18.505890    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-z6kzz" podStartSLOduration=1.009598068 podStartE2EDuration="1.505871083s" totalImagesPullingTime="496.273015ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:16:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:16:18.502569181 +0000 UTC m=+3106.467150267" watchObservedRunningTime="2026-07-30 20:16:18.505871083 +0000 UTC m=+3106.470452173"
Jul 30 20:16:27.750593 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:27.750554    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-z6kzz"
Jul 30 20:16:27.751057 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:27.750645    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-z6kzz"
Jul 30 20:16:27.772477 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:27.772449    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-z6kzz"
Jul 30 20:16:28.575287 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:28.575263    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-z6kzz"
Jul 30 20:16:28.607941 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:28.607910    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-z6kzz"]
Jul 30 20:16:30.565910 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:30.565865    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-z6kzz" podUID="a7637a53-8564-4cf7-9331-bc0fc216b4be" containerName="registry-server" containerID="cri-o://a994b093beea1fec5edf6c914eec6443064b176a3248f0d9067eda328f94b738" gracePeriod=2
Jul 30 20:16:30.818608 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:30.818550    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-z6kzz"
Jul 30 20:16:30.907154 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:30.907121    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zjcl\" (UniqueName: \"kubernetes.io/projected/a7637a53-8564-4cf7-9331-bc0fc216b4be-kube-api-access-2zjcl\") pod \"a7637a53-8564-4cf7-9331-bc0fc216b4be\" (UID: \"a7637a53-8564-4cf7-9331-bc0fc216b4be\") "
Jul 30 20:16:30.909276 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:30.909243    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7637a53-8564-4cf7-9331-bc0fc216b4be-kube-api-access-2zjcl" (OuterVolumeSpecName: "kube-api-access-2zjcl") pod "a7637a53-8564-4cf7-9331-bc0fc216b4be" (UID: "a7637a53-8564-4cf7-9331-bc0fc216b4be"). InnerVolumeSpecName "kube-api-access-2zjcl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:16:31.008667 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:31.008636    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zjcl\" (UniqueName: \"kubernetes.io/projected/a7637a53-8564-4cf7-9331-bc0fc216b4be-kube-api-access-2zjcl\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:16:31.574014 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:31.573980    2554 generic.go:356] "Generic (PLEG): container finished" podID="a7637a53-8564-4cf7-9331-bc0fc216b4be" containerID="a994b093beea1fec5edf6c914eec6443064b176a3248f0d9067eda328f94b738" exitCode=0
Jul 30 20:16:31.574438 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:31.574031    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-z6kzz"
Jul 30 20:16:31.574438 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:31.574041    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-z6kzz" event={"ID":"a7637a53-8564-4cf7-9331-bc0fc216b4be","Type":"ContainerDied","Data":"a994b093beea1fec5edf6c914eec6443064b176a3248f0d9067eda328f94b738"}
Jul 30 20:16:31.574438 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:31.574081    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-z6kzz" event={"ID":"a7637a53-8564-4cf7-9331-bc0fc216b4be","Type":"ContainerDied","Data":"ba2f01cee309fe0fa1f8d8a09eb002d33eb8d79781d8c2ac94f559669d3c8b2b"}
Jul 30 20:16:31.574438 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:31.574100    2554 scope.go:117] "RemoveContainer" containerID="a994b093beea1fec5edf6c914eec6443064b176a3248f0d9067eda328f94b738"
Jul 30 20:16:31.584355 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:31.584337    2554 scope.go:117] "RemoveContainer" containerID="a994b093beea1fec5edf6c914eec6443064b176a3248f0d9067eda328f94b738"
Jul 30 20:16:31.584608 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:16:31.584588    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a994b093beea1fec5edf6c914eec6443064b176a3248f0d9067eda328f94b738\": container with ID starting with a994b093beea1fec5edf6c914eec6443064b176a3248f0d9067eda328f94b738 not found: ID does not exist" containerID="a994b093beea1fec5edf6c914eec6443064b176a3248f0d9067eda328f94b738"
Jul 30 20:16:31.584655 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:31.584617    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a994b093beea1fec5edf6c914eec6443064b176a3248f0d9067eda328f94b738"} err="failed to get container status \"a994b093beea1fec5edf6c914eec6443064b176a3248f0d9067eda328f94b738\": rpc error: code = NotFound desc = could not find container \"a994b093beea1fec5edf6c914eec6443064b176a3248f0d9067eda328f94b738\": container with ID starting with a994b093beea1fec5edf6c914eec6443064b176a3248f0d9067eda328f94b738 not found: ID does not exist"
Jul 30 20:16:31.597519 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:31.597493    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-z6kzz"]
Jul 30 20:16:31.600007 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:31.599988    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-z6kzz"]
Jul 30 20:16:32.488093 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:32.488062    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7637a53-8564-4cf7-9331-bc0fc216b4be" path="/var/lib/kubelet/pods/a7637a53-8564-4cf7-9331-bc0fc216b4be/volumes"
Jul 30 20:16:33.009076 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:33.009046    2554 scope.go:117] "RemoveContainer" containerID="0af7a360a19efcdc920a604fc226ab9eca2b25b9a0f23a1652d8e73961b0e982"
Jul 30 20:16:42.319676 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:16:42.319641    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:17:01.517371 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:17:01.517316    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:17:01.618241 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:17:01.618200    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:17:33.039539 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:17:33.039502    2554 scope.go:117] "RemoveContainer" containerID="495419814c16d4d721008c10649732b634b83156d55fc65f5edab9b0f96b475f"
Jul 30 20:17:33.051187 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:17:33.051163    2554 scope.go:117] "RemoveContainer" containerID="80ad5c31c78234811d868b17ea15cbc02c5f9c03bb0d9f8d99fe74b93a99d5cb"
Jul 30 20:18:06.163197 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:06.163140    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:18:06.972581 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:06.972549    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"]
Jul 30 20:18:08.341769 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:08.341720    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:18:15.902621 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:15.902569    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:18:21.992043 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:21.991981    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc" podUID="a8e913fa-d34e-4134-90f0-7a08d632ec13" containerName="main" containerID="cri-o://3dd30580ad0b61c742966a6cb78aea3bd7233e2e4f7239e3b87f509aaf37f42c" gracePeriod=45
Jul 30 20:18:22.243682 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:22.243626    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"
Jul 30 20:18:22.244838 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:22.244814    2554 generic.go:356] "Generic (PLEG): container finished" podID="a8e913fa-d34e-4134-90f0-7a08d632ec13" containerID="3dd30580ad0b61c742966a6cb78aea3bd7233e2e4f7239e3b87f509aaf37f42c" exitCode=0
Jul 30 20:18:22.244934 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:22.244887    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc" event={"ID":"a8e913fa-d34e-4134-90f0-7a08d632ec13","Type":"ContainerDied","Data":"3dd30580ad0b61c742966a6cb78aea3bd7233e2e4f7239e3b87f509aaf37f42c"}
Jul 30 20:18:22.245004 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:22.244931    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc" event={"ID":"a8e913fa-d34e-4134-90f0-7a08d632ec13","Type":"ContainerDied","Data":"1f08a79db638e4b74b10e9180b8a226dd25b72d99cd2a4a9a5034f80ede6dc00"}
Jul 30 20:18:22.245004 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:22.244970    2554 scope.go:117] "RemoveContainer" containerID="3dd30580ad0b61c742966a6cb78aea3bd7233e2e4f7239e3b87f509aaf37f42c"
Jul 30 20:18:22.253390 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:22.253374    2554 scope.go:117] "RemoveContainer" containerID="3dd30580ad0b61c742966a6cb78aea3bd7233e2e4f7239e3b87f509aaf37f42c"
Jul 30 20:18:22.253635 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:18:22.253616    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dd30580ad0b61c742966a6cb78aea3bd7233e2e4f7239e3b87f509aaf37f42c\": container with ID starting with 3dd30580ad0b61c742966a6cb78aea3bd7233e2e4f7239e3b87f509aaf37f42c not found: ID does not exist" containerID="3dd30580ad0b61c742966a6cb78aea3bd7233e2e4f7239e3b87f509aaf37f42c"
Jul 30 20:18:22.253679 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:22.253643    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dd30580ad0b61c742966a6cb78aea3bd7233e2e4f7239e3b87f509aaf37f42c"} err="failed to get container status \"3dd30580ad0b61c742966a6cb78aea3bd7233e2e4f7239e3b87f509aaf37f42c\": rpc error: code = NotFound desc = could not find container \"3dd30580ad0b61c742966a6cb78aea3bd7233e2e4f7239e3b87f509aaf37f42c\": container with ID starting with 3dd30580ad0b61c742966a6cb78aea3bd7233e2e4f7239e3b87f509aaf37f42c not found: ID does not exist"
Jul 30 20:18:22.285812 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:22.285787    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8e913fa-d34e-4134-90f0-7a08d632ec13-home\") pod \"a8e913fa-d34e-4134-90f0-7a08d632ec13\" (UID: \"a8e913fa-d34e-4134-90f0-7a08d632ec13\") "
Jul 30 20:18:22.285931 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:22.285826    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8e913fa-d34e-4134-90f0-7a08d632ec13-tls-certs\") pod \"a8e913fa-d34e-4134-90f0-7a08d632ec13\" (UID: \"a8e913fa-d34e-4134-90f0-7a08d632ec13\") "
Jul 30 20:18:22.285931 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:22.285878    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8e913fa-d34e-4134-90f0-7a08d632ec13-tmp-dir\") pod \"a8e913fa-d34e-4134-90f0-7a08d632ec13\" (UID: \"a8e913fa-d34e-4134-90f0-7a08d632ec13\") "
Jul 30 20:18:22.285931 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:22.285908    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pj72\" (UniqueName: \"kubernetes.io/projected/a8e913fa-d34e-4134-90f0-7a08d632ec13-kube-api-access-8pj72\") pod \"a8e913fa-d34e-4134-90f0-7a08d632ec13\" (UID: \"a8e913fa-d34e-4134-90f0-7a08d632ec13\") "
Jul 30 20:18:22.286125 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:22.285978    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8e913fa-d34e-4134-90f0-7a08d632ec13-dshm\") pod \"a8e913fa-d34e-4134-90f0-7a08d632ec13\" (UID: \"a8e913fa-d34e-4134-90f0-7a08d632ec13\") "
Jul 30 20:18:22.286125 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:22.286017    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8e913fa-d34e-4134-90f0-7a08d632ec13-model-cache\") pod \"a8e913fa-d34e-4134-90f0-7a08d632ec13\" (UID: \"a8e913fa-d34e-4134-90f0-7a08d632ec13\") "
Jul 30 20:18:22.286125 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:22.286108    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8e913fa-d34e-4134-90f0-7a08d632ec13-home" (OuterVolumeSpecName: "home") pod "a8e913fa-d34e-4134-90f0-7a08d632ec13" (UID: "a8e913fa-d34e-4134-90f0-7a08d632ec13"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:18:22.286288 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:22.286195    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8e913fa-d34e-4134-90f0-7a08d632ec13-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a8e913fa-d34e-4134-90f0-7a08d632ec13" (UID: "a8e913fa-d34e-4134-90f0-7a08d632ec13"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:18:22.286419 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:22.286393    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8e913fa-d34e-4134-90f0-7a08d632ec13-model-cache" (OuterVolumeSpecName: "model-cache") pod "a8e913fa-d34e-4134-90f0-7a08d632ec13" (UID: "a8e913fa-d34e-4134-90f0-7a08d632ec13"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:18:22.286419 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:22.286409    2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8e913fa-d34e-4134-90f0-7a08d632ec13-tmp-dir\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:18:22.286580 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:22.286429    2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8e913fa-d34e-4134-90f0-7a08d632ec13-home\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:18:22.288214 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:22.288193    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8e913fa-d34e-4134-90f0-7a08d632ec13-kube-api-access-8pj72" (OuterVolumeSpecName: "kube-api-access-8pj72") pod "a8e913fa-d34e-4134-90f0-7a08d632ec13" (UID: "a8e913fa-d34e-4134-90f0-7a08d632ec13"). InnerVolumeSpecName "kube-api-access-8pj72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:18:22.288291 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:22.288240    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8e913fa-d34e-4134-90f0-7a08d632ec13-dshm" (OuterVolumeSpecName: "dshm") pod "a8e913fa-d34e-4134-90f0-7a08d632ec13" (UID: "a8e913fa-d34e-4134-90f0-7a08d632ec13"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:18:22.288388 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:22.288372    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8e913fa-d34e-4134-90f0-7a08d632ec13-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a8e913fa-d34e-4134-90f0-7a08d632ec13" (UID: "a8e913fa-d34e-4134-90f0-7a08d632ec13"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:18:22.387509 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:22.387475    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pj72\" (UniqueName: \"kubernetes.io/projected/a8e913fa-d34e-4134-90f0-7a08d632ec13-kube-api-access-8pj72\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:18:22.387509 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:22.387503    2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8e913fa-d34e-4134-90f0-7a08d632ec13-dshm\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:18:22.387509 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:22.387513    2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8e913fa-d34e-4134-90f0-7a08d632ec13-model-cache\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:18:22.387750 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:22.387525    2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8e913fa-d34e-4134-90f0-7a08d632ec13-tls-certs\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:18:23.251267 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:23.251239    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"
Jul 30 20:18:23.268302 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:23.268269    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"]
Jul 30 20:18:23.276798 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:23.276772    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-a769e5/auth-test-model-a769e5-kserve-7bb8894895-j6fkc"]
Jul 30 20:18:24.487082 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:24.487011    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8e913fa-d34e-4134-90f0-7a08d632ec13" path="/var/lib/kubelet/pods/a8e913fa-d34e-4134-90f0-7a08d632ec13/volumes"
Jul 30 20:18:47.241522 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:47.241490    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:18:53.572380 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.572344    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"]
Jul 30 20:18:53.572974 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.572867    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8e913fa-d34e-4134-90f0-7a08d632ec13" containerName="main"
Jul 30 20:18:53.572974 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.572886    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8e913fa-d34e-4134-90f0-7a08d632ec13" containerName="main"
Jul 30 20:18:53.572974 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.572906    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7637a53-8564-4cf7-9331-bc0fc216b4be" containerName="registry-server"
Jul 30 20:18:53.572974 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.572913    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7637a53-8564-4cf7-9331-bc0fc216b4be" containerName="registry-server"
Jul 30 20:18:53.573162 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.573017    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8e913fa-d34e-4134-90f0-7a08d632ec13" containerName="main"
Jul 30 20:18:53.573162 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.573031    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7637a53-8564-4cf7-9331-bc0fc216b4be" containerName="registry-server"
Jul 30 20:18:53.575921 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.575906    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"
Jul 30 20:18:53.579573 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.579548    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a769e5\"/\"kube-root-ca.crt\""
Jul 30 20:18:53.579689 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.579556    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a769e5\"/\"sub-test-model-a769e5-kserve-self-signed-certs\""
Jul 30 20:18:53.579689 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.579610    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a769e5\"/\"default-dockercfg-gprzj\""
Jul 30 20:18:53.579689 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.579556    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a769e5\"/\"openshift-service-ca.crt\""
Jul 30 20:18:53.584927 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.584907    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"]
Jul 30 20:18:53.588347 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.588320    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:18:53.694736 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.694699    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-tls-certs\") pod \"sub-test-model-a769e5-kserve-6f467564f7-nwr75\" (UID: \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\") " pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"
Jul 30 20:18:53.694736 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.694735    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-dshm\") pod \"sub-test-model-a769e5-kserve-6f467564f7-nwr75\" (UID: \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\") " pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"
Jul 30 20:18:53.694968 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.694750    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-tmp-dir\") pod \"sub-test-model-a769e5-kserve-6f467564f7-nwr75\" (UID: \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\") " pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"
Jul 30 20:18:53.694968 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.694838    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-home\") pod \"sub-test-model-a769e5-kserve-6f467564f7-nwr75\" (UID: \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\") " pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"
Jul 30 20:18:53.694968 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.694876    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgbj5\" (UniqueName: \"kubernetes.io/projected/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-kube-api-access-sgbj5\") pod \"sub-test-model-a769e5-kserve-6f467564f7-nwr75\" (UID: \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\") " pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"
Jul 30 20:18:53.694968 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.694924    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-model-cache\") pod \"sub-test-model-a769e5-kserve-6f467564f7-nwr75\" (UID: \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\") " pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"
Jul 30 20:18:53.795896 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.795864    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-model-cache\") pod \"sub-test-model-a769e5-kserve-6f467564f7-nwr75\" (UID: \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\") " pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"
Jul 30 20:18:53.796082 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.795946    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-tls-certs\") pod \"sub-test-model-a769e5-kserve-6f467564f7-nwr75\" (UID: \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\") " pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"
Jul 30 20:18:53.796082 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.795984    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-dshm\") pod \"sub-test-model-a769e5-kserve-6f467564f7-nwr75\" (UID: \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\") " pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"
Jul 30 20:18:53.796082 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.796001    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-tmp-dir\") pod \"sub-test-model-a769e5-kserve-6f467564f7-nwr75\" (UID: \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\") " pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"
Jul 30 20:18:53.796082 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.796035    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-home\") pod \"sub-test-model-a769e5-kserve-6f467564f7-nwr75\" (UID: \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\") " pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"
Jul 30 20:18:53.796082 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.796064    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgbj5\" (UniqueName: \"kubernetes.io/projected/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-kube-api-access-sgbj5\") pod \"sub-test-model-a769e5-kserve-6f467564f7-nwr75\" (UID: \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\") " pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"
Jul 30 20:18:53.796359 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.796336    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-model-cache\") pod \"sub-test-model-a769e5-kserve-6f467564f7-nwr75\" (UID: \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\") " pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"
Jul 30 20:18:53.796419 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.796375    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-home\") pod \"sub-test-model-a769e5-kserve-6f467564f7-nwr75\" (UID: \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\") " pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"
Jul 30 20:18:53.796466 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.796430    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-tmp-dir\") pod \"sub-test-model-a769e5-kserve-6f467564f7-nwr75\" (UID: \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\") " pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"
Jul 30 20:18:53.798264 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.798244    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-dshm\") pod \"sub-test-model-a769e5-kserve-6f467564f7-nwr75\" (UID: \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\") " pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"
Jul 30 20:18:53.798535 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.798519    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-tls-certs\") pod \"sub-test-model-a769e5-kserve-6f467564f7-nwr75\" (UID: \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\") " pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"
Jul 30 20:18:53.804823 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.804800    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgbj5\" (UniqueName: \"kubernetes.io/projected/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-kube-api-access-sgbj5\") pod \"sub-test-model-a769e5-kserve-6f467564f7-nwr75\" (UID: \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\") " pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"
Jul 30 20:18:53.886685 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:53.886600    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"
Jul 30 20:18:54.023906 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:54.023873    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"]
Jul 30 20:18:54.025362 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:18:54.025333    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b9112bb_dc98_49fd_8ab5_5ef4bd4ba936.slice/crio-8763d5993f3ee2422d904a010e1a9aa0af5fbc536b6ce46975b6438ededc7a51 WatchSource:0}: Error finding container 8763d5993f3ee2422d904a010e1a9aa0af5fbc536b6ce46975b6438ededc7a51: Status 404 returned error can't find the container with id 8763d5993f3ee2422d904a010e1a9aa0af5fbc536b6ce46975b6438ededc7a51
Jul 30 20:18:54.432987 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:54.432930    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75" event={"ID":"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936","Type":"ContainerStarted","Data":"309d833d524c7e5a68049ab3f7a9b4f61ab21261950d9775645ce7bc20190d07"}
Jul 30 20:18:54.433164 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:54.432998    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75" event={"ID":"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936","Type":"ContainerStarted","Data":"8763d5993f3ee2422d904a010e1a9aa0af5fbc536b6ce46975b6438ededc7a51"}
Jul 30 20:18:54.451706 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:18:54.451650    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75" podStartSLOduration=1.451635515 podStartE2EDuration="1.451635515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:18:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:18:54.449776678 +0000 UTC m=+3262.414357758" watchObservedRunningTime="2026-07-30 20:18:54.451635515 +0000 UTC m=+3262.416216593"
Jul 30 20:19:03.887011 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:19:03.886977    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"
Jul 30 20:19:03.887496 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:19:03.887031    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"
Jul 30 20:19:03.899695 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:19:03.899670    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"
Jul 30 20:19:04.500900 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:19:04.500865    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"
Jul 30 20:19:32.843752 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:19:32.843713    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/1.log"
Jul 30 20:19:32.848474 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:19:32.848448    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 20:19:32.852283 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:19:32.852264    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/1.log"
Jul 30 20:19:32.856601 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:19:32.856584    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 20:20:33.227604 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:20:33.227512    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:20:43.668579 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:20:43.668542    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:20:44.557404 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:20:44.557362    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"]
Jul 30 20:20:44.568340 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:20:44.568306    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:20:55.903188 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:20:55.903148    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:20:59.576871 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:20:59.576829    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75" podUID="7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936" containerName="main" containerID="cri-o://309d833d524c7e5a68049ab3f7a9b4f61ab21261950d9775645ce7bc20190d07" gracePeriod=45
Jul 30 20:20:59.837107 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:20:59.837045    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"
Jul 30 20:20:59.934199 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:20:59.934163    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-home\") pod \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\" (UID: \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\") "
Jul 30 20:20:59.934381 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:20:59.934208    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-dshm\") pod \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\" (UID: \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\") "
Jul 30 20:20:59.934381 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:20:59.934251    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgbj5\" (UniqueName: \"kubernetes.io/projected/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-kube-api-access-sgbj5\") pod \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\" (UID: \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\") "
Jul 30 20:20:59.934381 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:20:59.934301    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-tls-certs\") pod \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\" (UID: \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\") "
Jul 30 20:20:59.934381 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:20:59.934338    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-tmp-dir\") pod \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\" (UID: \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\") "
Jul 30 20:20:59.934590 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:20:59.934382    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-model-cache\") pod \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\" (UID: \"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936\") "
Jul 30 20:20:59.934590 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:20:59.934436    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-home" (OuterVolumeSpecName: "home") pod "7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936" (UID: "7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:20:59.934686 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:20:59.934602    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936" (UID: "7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:20:59.934734 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:20:59.934699    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-model-cache" (OuterVolumeSpecName: "model-cache") pod "7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936" (UID: "7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:20:59.934773 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:20:59.934756    2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-home\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:20:59.934818 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:20:59.934787    2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-tmp-dir\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:20:59.936425 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:20:59.936399    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936" (UID: "7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:20:59.936824 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:20:59.936801    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-dshm" (OuterVolumeSpecName: "dshm") pod "7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936" (UID: "7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:20:59.936884 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:20:59.936823    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-kube-api-access-sgbj5" (OuterVolumeSpecName: "kube-api-access-sgbj5") pod "7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936" (UID: "7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936"). InnerVolumeSpecName "kube-api-access-sgbj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:21:00.036321 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:21:00.036282    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgbj5\" (UniqueName: \"kubernetes.io/projected/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-kube-api-access-sgbj5\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:21:00.036321 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:21:00.036315    2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-tls-certs\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:21:00.036321 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:21:00.036324    2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-model-cache\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:21:00.036321 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:21:00.036332    2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936-dshm\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:21:00.191866 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:21:00.191763    2554 generic.go:356] "Generic (PLEG): container finished" podID="7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936" containerID="309d833d524c7e5a68049ab3f7a9b4f61ab21261950d9775645ce7bc20190d07" exitCode=0
Jul 30 20:21:00.191866 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:21:00.191812    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"
Jul 30 20:21:00.192120 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:21:00.191862    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75" event={"ID":"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936","Type":"ContainerDied","Data":"309d833d524c7e5a68049ab3f7a9b4f61ab21261950d9775645ce7bc20190d07"}
Jul 30 20:21:00.192120 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:21:00.191902    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75" event={"ID":"7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936","Type":"ContainerDied","Data":"8763d5993f3ee2422d904a010e1a9aa0af5fbc536b6ce46975b6438ededc7a51"}
Jul 30 20:21:00.192120 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:21:00.191919    2554 scope.go:117] "RemoveContainer" containerID="309d833d524c7e5a68049ab3f7a9b4f61ab21261950d9775645ce7bc20190d07"
Jul 30 20:21:00.201521 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:21:00.201506    2554 scope.go:117] "RemoveContainer" containerID="309d833d524c7e5a68049ab3f7a9b4f61ab21261950d9775645ce7bc20190d07"
Jul 30 20:21:00.201743 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:21:00.201726    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"309d833d524c7e5a68049ab3f7a9b4f61ab21261950d9775645ce7bc20190d07\": container with ID starting with 309d833d524c7e5a68049ab3f7a9b4f61ab21261950d9775645ce7bc20190d07 not found: ID does not exist" containerID="309d833d524c7e5a68049ab3f7a9b4f61ab21261950d9775645ce7bc20190d07"
Jul 30 20:21:00.201796 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:21:00.201750    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"309d833d524c7e5a68049ab3f7a9b4f61ab21261950d9775645ce7bc20190d07"} err="failed to get container status \"309d833d524c7e5a68049ab3f7a9b4f61ab21261950d9775645ce7bc20190d07\": rpc error: code = NotFound desc = could not find container \"309d833d524c7e5a68049ab3f7a9b4f61ab21261950d9775645ce7bc20190d07\": container with ID starting with 309d833d524c7e5a68049ab3f7a9b4f61ab21261950d9775645ce7bc20190d07 not found: ID does not exist"
Jul 30 20:21:00.214579 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:21:00.214548    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"]
Jul 30 20:21:00.218152 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:21:00.218123    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-a769e5/sub-test-model-a769e5-kserve-6f467564f7-nwr75"]
Jul 30 20:21:00.488534 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:21:00.488500    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936" path="/var/lib/kubelet/pods/7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936/volumes"
Jul 30 20:21:16.500644 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:21:16.500605    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:21:18.031129 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:21:18.031088    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:21:18.129778 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:21:18.129736    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:21:23.948413 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:21:23.948373    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:21:28.030883 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:21:28.030791    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:23:04.446166 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:04.446080    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:23:05.299723 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:05.299685    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:23:06.258663 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:06.258626    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7730-140683917126192"]
Jul 30 20:23:06.259119 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:06.259102    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936" containerName="main"
Jul 30 20:23:06.259171 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:06.259122    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936" containerName="main"
Jul 30 20:23:06.259212 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:06.259204    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b9112bb-dc98-49fd-8ab5-5ef4bd4ba936" containerName="main"
Jul 30 20:23:06.262134 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:06.262118    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-140683917126192"
Jul 30 20:23:06.264638 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:06.264620    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4wwrf\""
Jul 30 20:23:06.268910 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:06.268890    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7730-140683917126192"]
Jul 30 20:23:06.351323 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:06.351275    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdc8v\" (UniqueName: \"kubernetes.io/projected/42674561-c669-45d7-b995-8479c654af6d-kube-api-access-fdc8v\") pod \"test-curl-7730-140683917126192\" (UID: \"42674561-c669-45d7-b995-8479c654af6d\") " pod="opendatahub/test-curl-7730-140683917126192"
Jul 30 20:23:06.452795 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:06.452752    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdc8v\" (UniqueName: \"kubernetes.io/projected/42674561-c669-45d7-b995-8479c654af6d-kube-api-access-fdc8v\") pod \"test-curl-7730-140683917126192\" (UID: \"42674561-c669-45d7-b995-8479c654af6d\") " pod="opendatahub/test-curl-7730-140683917126192"
Jul 30 20:23:06.460894 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:06.460867    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdc8v\" (UniqueName: \"kubernetes.io/projected/42674561-c669-45d7-b995-8479c654af6d-kube-api-access-fdc8v\") pod \"test-curl-7730-140683917126192\" (UID: \"42674561-c669-45d7-b995-8479c654af6d\") " pod="opendatahub/test-curl-7730-140683917126192"
Jul 30 20:23:06.573035 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:06.572929    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-140683917126192"
Jul 30 20:23:06.714502 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:06.714473    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7730-140683917126192"]
Jul 30 20:23:06.716092 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:23:06.716066    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod42674561_c669_45d7_b995_8479c654af6d.slice/crio-c69c36a63b0a8e733ca3c40f3771002cec48721f6f07d44768b0e55f0e753a3a WatchSource:0}: Error finding container c69c36a63b0a8e733ca3c40f3771002cec48721f6f07d44768b0e55f0e753a3a: Status 404 returned error can't find the container with id c69c36a63b0a8e733ca3c40f3771002cec48721f6f07d44768b0e55f0e753a3a
Jul 30 20:23:06.717844 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:06.717826    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 20:23:06.941730 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:06.941645    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7730-140683917126192" event={"ID":"42674561-c669-45d7-b995-8479c654af6d","Type":"ContainerStarted","Data":"c69c36a63b0a8e733ca3c40f3771002cec48721f6f07d44768b0e55f0e753a3a"}
Jul 30 20:23:07.949897 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:07.949862    2554 generic.go:356] "Generic (PLEG): container finished" podID="42674561-c669-45d7-b995-8479c654af6d" containerID="9e19047da97c37c3d1420af788b8c5ca43279de75710dd7749f44ab77a40b4cd" exitCode=0
Jul 30 20:23:07.950325 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:07.949963    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7730-140683917126192" event={"ID":"42674561-c669-45d7-b995-8479c654af6d","Type":"ContainerDied","Data":"9e19047da97c37c3d1420af788b8c5ca43279de75710dd7749f44ab77a40b4cd"}
Jul 30 20:23:09.104236 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:09.104208    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-140683917126192"
Jul 30 20:23:09.123899 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:09.123875    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7730-140683917126192_42674561-c669-45d7-b995-8479c654af6d/test-curl-7730-140683917126192/0.log"
Jul 30 20:23:09.148193 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:09.148160    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7730-140683917126192"]
Jul 30 20:23:09.153428 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:09.153389    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7730-140683917126192"]
Jul 30 20:23:09.274786 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:09.274757    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7730-140683917133104"]
Jul 30 20:23:09.275180 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:09.275166    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42674561-c669-45d7-b995-8479c654af6d" containerName="test-curl-7730-140683917126192"
Jul 30 20:23:09.275237 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:09.275182    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="42674561-c669-45d7-b995-8479c654af6d" containerName="test-curl-7730-140683917126192"
Jul 30 20:23:09.275278 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:09.275260    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="42674561-c669-45d7-b995-8479c654af6d" containerName="test-curl-7730-140683917126192"
Jul 30 20:23:09.278276 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:09.278260    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-140683917133104"
Jul 30 20:23:09.281940 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:09.281916    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdc8v\" (UniqueName: \"kubernetes.io/projected/42674561-c669-45d7-b995-8479c654af6d-kube-api-access-fdc8v\") pod \"42674561-c669-45d7-b995-8479c654af6d\" (UID: \"42674561-c669-45d7-b995-8479c654af6d\") "
Jul 30 20:23:09.284753 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:09.284725    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42674561-c669-45d7-b995-8479c654af6d-kube-api-access-fdc8v" (OuterVolumeSpecName: "kube-api-access-fdc8v") pod "42674561-c669-45d7-b995-8479c654af6d" (UID: "42674561-c669-45d7-b995-8479c654af6d"). InnerVolumeSpecName "kube-api-access-fdc8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:23:09.288546 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:09.286785    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7730-140683917133104"]
Jul 30 20:23:09.383911 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:09.383862    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lngqc\" (UniqueName: \"kubernetes.io/projected/8b475630-b31a-4909-b34c-869b1785b36b-kube-api-access-lngqc\") pod \"test-curl-7730-140683917133104\" (UID: \"8b475630-b31a-4909-b34c-869b1785b36b\") " pod="opendatahub/test-curl-7730-140683917133104"
Jul 30 20:23:09.384113 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:09.384026    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdc8v\" (UniqueName: \"kubernetes.io/projected/42674561-c669-45d7-b995-8479c654af6d-kube-api-access-fdc8v\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:23:09.485419 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:09.485385    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lngqc\" (UniqueName: \"kubernetes.io/projected/8b475630-b31a-4909-b34c-869b1785b36b-kube-api-access-lngqc\") pod \"test-curl-7730-140683917133104\" (UID: \"8b475630-b31a-4909-b34c-869b1785b36b\") " pod="opendatahub/test-curl-7730-140683917133104"
Jul 30 20:23:09.493820 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:09.493789    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lngqc\" (UniqueName: \"kubernetes.io/projected/8b475630-b31a-4909-b34c-869b1785b36b-kube-api-access-lngqc\") pod \"test-curl-7730-140683917133104\" (UID: \"8b475630-b31a-4909-b34c-869b1785b36b\") " pod="opendatahub/test-curl-7730-140683917133104"
Jul 30 20:23:09.602392 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:09.602308    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-140683917133104"
Jul 30 20:23:09.739328 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:09.739296    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7730-140683917133104"]
Jul 30 20:23:09.740361 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:23:09.740334    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b475630_b31a_4909_b34c_869b1785b36b.slice/crio-34cdb9b5b99bb832e32244644e784417613f207b4eebf0140e6e4547b8bb98f8 WatchSource:0}: Error finding container 34cdb9b5b99bb832e32244644e784417613f207b4eebf0140e6e4547b8bb98f8: Status 404 returned error can't find the container with id 34cdb9b5b99bb832e32244644e784417613f207b4eebf0140e6e4547b8bb98f8
Jul 30 20:23:09.962187 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:09.962161    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-140683917126192"
Jul 30 20:23:09.962187 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:09.962186    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c69c36a63b0a8e733ca3c40f3771002cec48721f6f07d44768b0e55f0e753a3a"
Jul 30 20:23:09.963340 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:09.963317    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7730-140683917133104" event={"ID":"8b475630-b31a-4909-b34c-869b1785b36b","Type":"ContainerStarted","Data":"34cdb9b5b99bb832e32244644e784417613f207b4eebf0140e6e4547b8bb98f8"}
Jul 30 20:23:09.973702 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:23:09.973669    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7730-140683917126192\" is forbidden: User \"system:node:ip-10-0-134-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-180.ec2.internal' and this object" podUID="42674561-c669-45d7-b995-8479c654af6d" pod="opendatahub/test-curl-7730-140683917126192"
Jul 30 20:23:10.487207 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:10.487176    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42674561-c669-45d7-b995-8479c654af6d" path="/var/lib/kubelet/pods/42674561-c669-45d7-b995-8479c654af6d/volumes"
Jul 30 20:23:10.971185 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:10.971152    2554 generic.go:356] "Generic (PLEG): container finished" podID="8b475630-b31a-4909-b34c-869b1785b36b" containerID="00416cb25fe0228d093d565d2833cdb2a4b67c407c61d4518ad8711a90c3e655" exitCode=0
Jul 30 20:23:10.971402 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:10.971212    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7730-140683917133104" event={"ID":"8b475630-b31a-4909-b34c-869b1785b36b","Type":"ContainerDied","Data":"00416cb25fe0228d093d565d2833cdb2a4b67c407c61d4518ad8711a90c3e655"}
Jul 30 20:23:12.110405 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:12.110376    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-140683917133104"
Jul 30 20:23:12.131662 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:12.131637    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7730-140683917133104_8b475630-b31a-4909-b34c-869b1785b36b/test-curl-7730-140683917133104/0.log"
Jul 30 20:23:12.155143 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:12.155116    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7730-140683917133104"]
Jul 30 20:23:12.158518 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:12.158494    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7730-140683917133104"]
Jul 30 20:23:12.211616 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:12.211589    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lngqc\" (UniqueName: \"kubernetes.io/projected/8b475630-b31a-4909-b34c-869b1785b36b-kube-api-access-lngqc\") pod \"8b475630-b31a-4909-b34c-869b1785b36b\" (UID: \"8b475630-b31a-4909-b34c-869b1785b36b\") "
Jul 30 20:23:12.213807 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:12.213775    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b475630-b31a-4909-b34c-869b1785b36b-kube-api-access-lngqc" (OuterVolumeSpecName: "kube-api-access-lngqc") pod "8b475630-b31a-4909-b34c-869b1785b36b" (UID: "8b475630-b31a-4909-b34c-869b1785b36b"). InnerVolumeSpecName "kube-api-access-lngqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:23:12.312441 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:12.312343    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7730-140683917069456"]
Jul 30 20:23:12.312819 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:12.312802    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b475630-b31a-4909-b34c-869b1785b36b" containerName="test-curl-7730-140683917133104"
Jul 30 20:23:12.312819 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:12.312819    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b475630-b31a-4909-b34c-869b1785b36b" containerName="test-curl-7730-140683917133104"
Jul 30 20:23:12.312929 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:12.312830    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lngqc\" (UniqueName: \"kubernetes.io/projected/8b475630-b31a-4909-b34c-869b1785b36b-kube-api-access-lngqc\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:23:12.312929 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:12.312895    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b475630-b31a-4909-b34c-869b1785b36b" containerName="test-curl-7730-140683917133104"
Jul 30 20:23:12.316129 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:12.316114    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-140683917069456"
Jul 30 20:23:12.321395 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:12.321369    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7730-140683917069456"]
Jul 30 20:23:12.414292 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:12.414255    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxx7m\" (UniqueName: \"kubernetes.io/projected/94e028ef-934e-46ea-a770-3bd455e62fbe-kube-api-access-vxx7m\") pod \"test-curl-7730-140683917069456\" (UID: \"94e028ef-934e-46ea-a770-3bd455e62fbe\") " pod="opendatahub/test-curl-7730-140683917069456"
Jul 30 20:23:12.488172 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:12.488136    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b475630-b31a-4909-b34c-869b1785b36b" path="/var/lib/kubelet/pods/8b475630-b31a-4909-b34c-869b1785b36b/volumes"
Jul 30 20:23:12.515728 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:12.515700    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxx7m\" (UniqueName: \"kubernetes.io/projected/94e028ef-934e-46ea-a770-3bd455e62fbe-kube-api-access-vxx7m\") pod \"test-curl-7730-140683917069456\" (UID: \"94e028ef-934e-46ea-a770-3bd455e62fbe\") " pod="opendatahub/test-curl-7730-140683917069456"
Jul 30 20:23:12.523795 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:12.523773    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxx7m\" (UniqueName: \"kubernetes.io/projected/94e028ef-934e-46ea-a770-3bd455e62fbe-kube-api-access-vxx7m\") pod \"test-curl-7730-140683917069456\" (UID: \"94e028ef-934e-46ea-a770-3bd455e62fbe\") " pod="opendatahub/test-curl-7730-140683917069456"
Jul 30 20:23:12.627199 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:12.627096    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-140683917069456"
Jul 30 20:23:12.774408 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:12.774379    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7730-140683917069456"]
Jul 30 20:23:12.777061 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:23:12.777033    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod94e028ef_934e_46ea_a770_3bd455e62fbe.slice/crio-a2d31244d9c69b14f9a3413b84f23731887fb6a85e78e4ed5be22a1224f27422 WatchSource:0}: Error finding container a2d31244d9c69b14f9a3413b84f23731887fb6a85e78e4ed5be22a1224f27422: Status 404 returned error can't find the container with id a2d31244d9c69b14f9a3413b84f23731887fb6a85e78e4ed5be22a1224f27422
Jul 30 20:23:12.984697 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:12.984647    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-140683917133104"
Jul 30 20:23:12.984697 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:12.984698    2554 scope.go:117] "RemoveContainer" containerID="00416cb25fe0228d093d565d2833cdb2a4b67c407c61d4518ad8711a90c3e655"
Jul 30 20:23:12.986104 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:12.986075    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7730-140683917069456" event={"ID":"94e028ef-934e-46ea-a770-3bd455e62fbe","Type":"ContainerStarted","Data":"a2d31244d9c69b14f9a3413b84f23731887fb6a85e78e4ed5be22a1224f27422"}
Jul 30 20:23:13.994097 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:13.994060    2554 generic.go:356] "Generic (PLEG): container finished" podID="94e028ef-934e-46ea-a770-3bd455e62fbe" containerID="b9bba5c5c627d525c783cfdc897ea885489d3104449f164fd27cad493e4eb135" exitCode=0
Jul 30 20:23:13.994478 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:13.994150    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7730-140683917069456" event={"ID":"94e028ef-934e-46ea-a770-3bd455e62fbe","Type":"ContainerDied","Data":"b9bba5c5c627d525c783cfdc897ea885489d3104449f164fd27cad493e4eb135"}
Jul 30 20:23:15.143833 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:15.143809    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-140683917069456"
Jul 30 20:23:15.162426 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:15.162400    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7730-140683917069456_94e028ef-934e-46ea-a770-3bd455e62fbe/test-curl-7730-140683917069456/0.log"
Jul 30 20:23:15.185580 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:15.185535    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7730-140683917069456"]
Jul 30 20:23:15.188914 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:15.188889    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7730-140683917069456"]
Jul 30 20:23:15.241637 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:15.241605    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxx7m\" (UniqueName: \"kubernetes.io/projected/94e028ef-934e-46ea-a770-3bd455e62fbe-kube-api-access-vxx7m\") pod \"94e028ef-934e-46ea-a770-3bd455e62fbe\" (UID: \"94e028ef-934e-46ea-a770-3bd455e62fbe\") "
Jul 30 20:23:15.243676 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:15.243647    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94e028ef-934e-46ea-a770-3bd455e62fbe-kube-api-access-vxx7m" (OuterVolumeSpecName: "kube-api-access-vxx7m") pod "94e028ef-934e-46ea-a770-3bd455e62fbe" (UID: "94e028ef-934e-46ea-a770-3bd455e62fbe"). InnerVolumeSpecName "kube-api-access-vxx7m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:23:15.302742 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:15.302667    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7730-140683917069456"]
Jul 30 20:23:15.303176 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:15.303157    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94e028ef-934e-46ea-a770-3bd455e62fbe" containerName="test-curl-7730-140683917069456"
Jul 30 20:23:15.303176 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:15.303177    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="94e028ef-934e-46ea-a770-3bd455e62fbe" containerName="test-curl-7730-140683917069456"
Jul 30 20:23:15.303294 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:15.303252    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="94e028ef-934e-46ea-a770-3bd455e62fbe" containerName="test-curl-7730-140683917069456"
Jul 30 20:23:15.306269 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:15.306250    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-140683917069456"
Jul 30 20:23:15.313489 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:15.313451    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7730-140683917069456"]
Jul 30 20:23:15.342705 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:15.342671    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxx7m\" (UniqueName: \"kubernetes.io/projected/94e028ef-934e-46ea-a770-3bd455e62fbe-kube-api-access-vxx7m\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:23:15.443693 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:15.443653    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlxkh\" (UniqueName: \"kubernetes.io/projected/8dc693ed-3c0e-4d77-8631-02e960b95bc1-kube-api-access-xlxkh\") pod \"test-curl-7730-140683917069456\" (UID: \"8dc693ed-3c0e-4d77-8631-02e960b95bc1\") " pod="opendatahub/test-curl-7730-140683917069456"
Jul 30 20:23:15.544686 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:15.544649    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlxkh\" (UniqueName: \"kubernetes.io/projected/8dc693ed-3c0e-4d77-8631-02e960b95bc1-kube-api-access-xlxkh\") pod \"test-curl-7730-140683917069456\" (UID: \"8dc693ed-3c0e-4d77-8631-02e960b95bc1\") " pod="opendatahub/test-curl-7730-140683917069456"
Jul 30 20:23:15.552887 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:15.552820    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlxkh\" (UniqueName: \"kubernetes.io/projected/8dc693ed-3c0e-4d77-8631-02e960b95bc1-kube-api-access-xlxkh\") pod \"test-curl-7730-140683917069456\" (UID: \"8dc693ed-3c0e-4d77-8631-02e960b95bc1\") " pod="opendatahub/test-curl-7730-140683917069456"
Jul 30 20:23:15.617538 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:15.617500    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-140683917069456"
Jul 30 20:23:15.755496 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:15.755471    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7730-140683917069456"]
Jul 30 20:23:15.757242 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:23:15.757209    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8dc693ed_3c0e_4d77_8631_02e960b95bc1.slice/crio-43dcc8c1589186dee07512e624bdd2b38382f85f8d7eacf8e11bf8a7cb63cf63 WatchSource:0}: Error finding container 43dcc8c1589186dee07512e624bdd2b38382f85f8d7eacf8e11bf8a7cb63cf63: Status 404 returned error can't find the container with id 43dcc8c1589186dee07512e624bdd2b38382f85f8d7eacf8e11bf8a7cb63cf63
Jul 30 20:23:16.007191 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:16.007153    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7730-140683917069456" event={"ID":"8dc693ed-3c0e-4d77-8631-02e960b95bc1","Type":"ContainerStarted","Data":"43dcc8c1589186dee07512e624bdd2b38382f85f8d7eacf8e11bf8a7cb63cf63"}
Jul 30 20:23:16.008325 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:16.008298    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-140683917069456"
Jul 30 20:23:16.008325 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:16.008313    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a2d31244d9c69b14f9a3413b84f23731887fb6a85e78e4ed5be22a1224f27422"
Jul 30 20:23:16.020536 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:16.020503    2554 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7730-140683917069456" oldPodUID="94e028ef-934e-46ea-a770-3bd455e62fbe" podUID="8dc693ed-3c0e-4d77-8631-02e960b95bc1"
Jul 30 20:23:16.487564 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:16.487525    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94e028ef-934e-46ea-a770-3bd455e62fbe" path="/var/lib/kubelet/pods/94e028ef-934e-46ea-a770-3bd455e62fbe/volumes"
Jul 30 20:23:17.017669 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:17.017634    2554 generic.go:356] "Generic (PLEG): container finished" podID="8dc693ed-3c0e-4d77-8631-02e960b95bc1" containerID="bca1587203d40aca5902cee1faba6b430be39c5233c1b7ef6584aa7040ba1cf2" exitCode=0
Jul 30 20:23:17.017847 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:17.017725    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7730-140683917069456" event={"ID":"8dc693ed-3c0e-4d77-8631-02e960b95bc1","Type":"ContainerDied","Data":"bca1587203d40aca5902cee1faba6b430be39c5233c1b7ef6584aa7040ba1cf2"}
Jul 30 20:23:18.172493 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:18.172455    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-140683917069456"
Jul 30 20:23:18.192050 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:18.192009    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7730-140683917069456_8dc693ed-3c0e-4d77-8631-02e960b95bc1/test-curl-7730-140683917069456/0.log"
Jul 30 20:23:18.214619 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:18.214582    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7730-140683917069456"]
Jul 30 20:23:18.220054 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:18.220026    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7730-140683917069456"]
Jul 30 20:23:18.270593 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:18.270560    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlxkh\" (UniqueName: \"kubernetes.io/projected/8dc693ed-3c0e-4d77-8631-02e960b95bc1-kube-api-access-xlxkh\") pod \"8dc693ed-3c0e-4d77-8631-02e960b95bc1\" (UID: \"8dc693ed-3c0e-4d77-8631-02e960b95bc1\") "
Jul 30 20:23:18.272772 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:18.272742    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dc693ed-3c0e-4d77-8631-02e960b95bc1-kube-api-access-xlxkh" (OuterVolumeSpecName: "kube-api-access-xlxkh") pod "8dc693ed-3c0e-4d77-8631-02e960b95bc1" (UID: "8dc693ed-3c0e-4d77-8631-02e960b95bc1"). InnerVolumeSpecName "kube-api-access-xlxkh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:23:18.330960 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:18.330926    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7730-140683917180976"]
Jul 30 20:23:18.331426 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:18.331411    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dc693ed-3c0e-4d77-8631-02e960b95bc1" containerName="test-curl-7730-140683917069456"
Jul 30 20:23:18.331466 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:18.331428    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dc693ed-3c0e-4d77-8631-02e960b95bc1" containerName="test-curl-7730-140683917069456"
Jul 30 20:23:18.331527 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:18.331518    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dc693ed-3c0e-4d77-8631-02e960b95bc1" containerName="test-curl-7730-140683917069456"
Jul 30 20:23:18.334640 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:18.334617    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-140683917180976"
Jul 30 20:23:18.340001 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:18.339973    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7730-140683917180976"]
Jul 30 20:23:18.371806 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:18.371773    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbkg7\" (UniqueName: \"kubernetes.io/projected/cc120741-0a67-411d-b31e-aa7841fb11e1-kube-api-access-wbkg7\") pod \"test-curl-7730-140683917180976\" (UID: \"cc120741-0a67-411d-b31e-aa7841fb11e1\") " pod="opendatahub/test-curl-7730-140683917180976"
Jul 30 20:23:18.371963 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:18.371821    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlxkh\" (UniqueName: \"kubernetes.io/projected/8dc693ed-3c0e-4d77-8631-02e960b95bc1-kube-api-access-xlxkh\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:23:18.472887 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:18.472849    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbkg7\" (UniqueName: \"kubernetes.io/projected/cc120741-0a67-411d-b31e-aa7841fb11e1-kube-api-access-wbkg7\") pod \"test-curl-7730-140683917180976\" (UID: \"cc120741-0a67-411d-b31e-aa7841fb11e1\") " pod="opendatahub/test-curl-7730-140683917180976"
Jul 30 20:23:18.480837 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:18.480811    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbkg7\" (UniqueName: \"kubernetes.io/projected/cc120741-0a67-411d-b31e-aa7841fb11e1-kube-api-access-wbkg7\") pod \"test-curl-7730-140683917180976\" (UID: \"cc120741-0a67-411d-b31e-aa7841fb11e1\") " pod="opendatahub/test-curl-7730-140683917180976"
Jul 30 20:23:18.487850 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:18.487822    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dc693ed-3c0e-4d77-8631-02e960b95bc1" path="/var/lib/kubelet/pods/8dc693ed-3c0e-4d77-8631-02e960b95bc1/volumes"
Jul 30 20:23:18.645976 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:18.645860    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-140683917180976"
Jul 30 20:23:18.786502 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:18.786469    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7730-140683917180976"]
Jul 30 20:23:18.786846 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:23:18.786823    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc120741_0a67_411d_b31e_aa7841fb11e1.slice/crio-a6235c83a20e77de74cbf165b8144ff73c3f959d8ee46fb5b1e541af8a93fd71 WatchSource:0}: Error finding container a6235c83a20e77de74cbf165b8144ff73c3f959d8ee46fb5b1e541af8a93fd71: Status 404 returned error can't find the container with id a6235c83a20e77de74cbf165b8144ff73c3f959d8ee46fb5b1e541af8a93fd71
Jul 30 20:23:19.031668 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:19.031632    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7730-140683917180976" event={"ID":"cc120741-0a67-411d-b31e-aa7841fb11e1","Type":"ContainerStarted","Data":"a6235c83a20e77de74cbf165b8144ff73c3f959d8ee46fb5b1e541af8a93fd71"}
Jul 30 20:23:19.032887 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:19.032867    2554 scope.go:117] "RemoveContainer" containerID="bca1587203d40aca5902cee1faba6b430be39c5233c1b7ef6584aa7040ba1cf2"
Jul 30 20:23:19.033021 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:19.032895    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-140683917069456"
Jul 30 20:23:20.041207 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:20.041170    2554 generic.go:356] "Generic (PLEG): container finished" podID="cc120741-0a67-411d-b31e-aa7841fb11e1" containerID="3bbacd344dce7423cc15bc0f94147437a63091887c3a6d3ae9be92d6dda561e1" exitCode=0
Jul 30 20:23:20.041582 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:20.041250    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7730-140683917180976" event={"ID":"cc120741-0a67-411d-b31e-aa7841fb11e1","Type":"ContainerDied","Data":"3bbacd344dce7423cc15bc0f94147437a63091887c3a6d3ae9be92d6dda561e1"}
Jul 30 20:23:21.185374 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:21.185347    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-140683917180976"
Jul 30 20:23:21.202314 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:21.202287    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbkg7\" (UniqueName: \"kubernetes.io/projected/cc120741-0a67-411d-b31e-aa7841fb11e1-kube-api-access-wbkg7\") pod \"cc120741-0a67-411d-b31e-aa7841fb11e1\" (UID: \"cc120741-0a67-411d-b31e-aa7841fb11e1\") "
Jul 30 20:23:21.204663 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:21.204632    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc120741-0a67-411d-b31e-aa7841fb11e1-kube-api-access-wbkg7" (OuterVolumeSpecName: "kube-api-access-wbkg7") pod "cc120741-0a67-411d-b31e-aa7841fb11e1" (UID: "cc120741-0a67-411d-b31e-aa7841fb11e1"). InnerVolumeSpecName "kube-api-access-wbkg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:23:21.205999 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:21.205889    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7730-140683917180976_cc120741-0a67-411d-b31e-aa7841fb11e1/test-curl-7730-140683917180976/0.log"
Jul 30 20:23:21.230180 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:21.230152    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7730-140683917180976"]
Jul 30 20:23:21.234052 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:21.234029    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7730-140683917180976"]
Jul 30 20:23:21.303617 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:21.303540    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbkg7\" (UniqueName: \"kubernetes.io/projected/cc120741-0a67-411d-b31e-aa7841fb11e1-kube-api-access-wbkg7\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:23:22.054992 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:22.054948    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7730-140683917180976"
Jul 30 20:23:22.055200 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:22.054947    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a6235c83a20e77de74cbf165b8144ff73c3f959d8ee46fb5b1e541af8a93fd71"
Jul 30 20:23:22.064948 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:23:22.064916    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7730-140683917180976\" is forbidden: User \"system:node:ip-10-0-134-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-180.ec2.internal' and this object" podUID="cc120741-0a67-411d-b31e-aa7841fb11e1" pod="opendatahub/test-curl-7730-140683917180976"
Jul 30 20:23:22.485288 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:23:22.485254    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7730-140683917180976\" is forbidden: User \"system:node:ip-10-0-134-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-180.ec2.internal' and this object" podUID="cc120741-0a67-411d-b31e-aa7841fb11e1" pod="opendatahub/test-curl-7730-140683917180976"
Jul 30 20:23:22.487435 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:22.487403    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc120741-0a67-411d-b31e-aa7841fb11e1" path="/var/lib/kubelet/pods/cc120741-0a67-411d-b31e-aa7841fb11e1/volumes"
Jul 30 20:23:24.841776 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:24.841744    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:23:26.095515 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:26.095478    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:23:28.083110 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:28.083070    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:23:28.192745 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:28.192713    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:23:29.208702 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:29.208666    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:23:29.403282 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:29.403241    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:23:29.711553 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:29.711518    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:23:29.818358 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:29.818323    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:23:34.114167 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:34.114129    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:23:53.090663 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:53.090628    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:23:53.777534 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:53.777499    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:23:54.120851 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:54.120772    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:23:54.222099 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:54.222062    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:23:57.297136 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:57.297083    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:23:57.395427 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:57.395395    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:23:58.392236 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:58.392198    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v"]
Jul 30 20:23:58.392730 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:58.392654    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc120741-0a67-411d-b31e-aa7841fb11e1" containerName="test-curl-7730-140683917180976"
Jul 30 20:23:58.392730 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:58.392670    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc120741-0a67-411d-b31e-aa7841fb11e1" containerName="test-curl-7730-140683917180976"
Jul 30 20:23:58.392861 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:58.392745    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc120741-0a67-411d-b31e-aa7841fb11e1" containerName="test-curl-7730-140683917180976"
Jul 30 20:23:58.396137 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:58.396105    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v"
Jul 30 20:23:58.398580 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:58.398559    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-d67d52-serving-cert\""
Jul 30 20:23:58.407385 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:58.407360    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v"]
Jul 30 20:23:58.424641 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:58.424613    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:23:58.465303 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:58.465270    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-998jp\" (UniqueName: \"kubernetes.io/projected/b390329e-625b-4236-a493-adb9ed8061cd-kube-api-access-998jp\") pod \"maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v\" (UID: \"b390329e-625b-4236-a493-adb9ed8061cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v"
Jul 30 20:23:58.465485 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:58.465327    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b390329e-625b-4236-a493-adb9ed8061cd-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v\" (UID: \"b390329e-625b-4236-a493-adb9ed8061cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v"
Jul 30 20:23:58.465485 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:58.465442    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b390329e-625b-4236-a493-adb9ed8061cd-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v\" (UID: \"b390329e-625b-4236-a493-adb9ed8061cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v"
Jul 30 20:23:58.566412 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:58.566361    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b390329e-625b-4236-a493-adb9ed8061cd-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v\" (UID: \"b390329e-625b-4236-a493-adb9ed8061cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v"
Jul 30 20:23:58.566602 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:58.566461    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b390329e-625b-4236-a493-adb9ed8061cd-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v\" (UID: \"b390329e-625b-4236-a493-adb9ed8061cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v"
Jul 30 20:23:58.566602 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:58.566504    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-998jp\" (UniqueName: \"kubernetes.io/projected/b390329e-625b-4236-a493-adb9ed8061cd-kube-api-access-998jp\") pod \"maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v\" (UID: \"b390329e-625b-4236-a493-adb9ed8061cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v"
Jul 30 20:23:58.566769 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:23:58.566749    2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-serving-cert: secret "maas-api-e2e-ipp-b-d67d52-serving-cert" not found
Jul 30 20:23:58.566847 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:23:58.566834    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b390329e-625b-4236-a493-adb9ed8061cd-maas-api-tls podName:b390329e-625b-4236-a493-adb9ed8061cd nodeName:}" failed. No retries permitted until 2026-07-30 20:23:59.066813632 +0000 UTC m=+3567.031394708 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b390329e-625b-4236-a493-adb9ed8061cd-maas-api-tls") pod "maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v" (UID: "b390329e-625b-4236-a493-adb9ed8061cd") : secret "maas-api-e2e-ipp-b-d67d52-serving-cert" not found
Jul 30 20:23:58.567108 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:58.567087    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b390329e-625b-4236-a493-adb9ed8061cd-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v\" (UID: \"b390329e-625b-4236-a493-adb9ed8061cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v"
Jul 30 20:23:58.575067 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:58.575044    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-998jp\" (UniqueName: \"kubernetes.io/projected/b390329e-625b-4236-a493-adb9ed8061cd-kube-api-access-998jp\") pod \"maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v\" (UID: \"b390329e-625b-4236-a493-adb9ed8061cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v"
Jul 30 20:23:58.901207 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:58.901176    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:23:59.002059 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:59.002016    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:23:59.053024 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:59.052984    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc"]
Jul 30 20:23:59.056611 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:59.056590    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc"
Jul 30 20:23:59.059101 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:59.059081    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-d67d52\""
Jul 30 20:23:59.059298 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:59.059283    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-d67d52-dockercfg-76wx5\""
Jul 30 20:23:59.068278 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:59.068257    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc"]
Jul 30 20:23:59.071259 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:59.071236    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3ffd11c-0d7b-4529-8edf-0925972813fd-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc\" (UID: \"f3ffd11c-0d7b-4529-8edf-0925972813fd\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc"
Jul 30 20:23:59.071373 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:59.071355    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b390329e-625b-4236-a493-adb9ed8061cd-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v\" (UID: \"b390329e-625b-4236-a493-adb9ed8061cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v"
Jul 30 20:23:59.071444 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:59.071414    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9hfr\" (UniqueName: \"kubernetes.io/projected/f3ffd11c-0d7b-4529-8edf-0925972813fd-kube-api-access-k9hfr\") pod \"payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc\" (UID: \"f3ffd11c-0d7b-4529-8edf-0925972813fd\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc"
Jul 30 20:23:59.071894 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:59.071869    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:23:59.073781 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:59.073760    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b390329e-625b-4236-a493-adb9ed8061cd-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v\" (UID: \"b390329e-625b-4236-a493-adb9ed8061cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v"
Jul 30 20:23:59.172975 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:59.172860    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9hfr\" (UniqueName: \"kubernetes.io/projected/f3ffd11c-0d7b-4529-8edf-0925972813fd-kube-api-access-k9hfr\") pod \"payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc\" (UID: \"f3ffd11c-0d7b-4529-8edf-0925972813fd\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc"
Jul 30 20:23:59.172975 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:59.172932    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3ffd11c-0d7b-4529-8edf-0925972813fd-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc\" (UID: \"f3ffd11c-0d7b-4529-8edf-0925972813fd\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc"
Jul 30 20:23:59.173584 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:59.173565    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3ffd11c-0d7b-4529-8edf-0925972813fd-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc\" (UID: \"f3ffd11c-0d7b-4529-8edf-0925972813fd\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc"
Jul 30 20:23:59.181396 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:59.181370    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9hfr\" (UniqueName: \"kubernetes.io/projected/f3ffd11c-0d7b-4529-8edf-0925972813fd-kube-api-access-k9hfr\") pod \"payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc\" (UID: \"f3ffd11c-0d7b-4529-8edf-0925972813fd\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc"
Jul 30 20:23:59.309280 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:59.309248    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v"
Jul 30 20:23:59.367841 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:59.367765    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc"
Jul 30 20:23:59.461071 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:59.461039    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v"]
Jul 30 20:23:59.463070 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:23:59.463038    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb390329e_625b_4236_a493_adb9ed8061cd.slice/crio-80b0efc69ed452aaf4204274bb0c7153ae8d4981340b930661c5d8263549938f WatchSource:0}: Error finding container 80b0efc69ed452aaf4204274bb0c7153ae8d4981340b930661c5d8263549938f: Status 404 returned error can't find the container with id 80b0efc69ed452aaf4204274bb0c7153ae8d4981340b930661c5d8263549938f
Jul 30 20:23:59.526381 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:23:59.526347    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3ffd11c_0d7b_4529_8edf_0925972813fd.slice/crio-8619b5c96e592168a9ab74bb7b002f60ea7c23c347e233c095b3a94c3f0691cf WatchSource:0}: Error finding container 8619b5c96e592168a9ab74bb7b002f60ea7c23c347e233c095b3a94c3f0691cf: Status 404 returned error can't find the container with id 8619b5c96e592168a9ab74bb7b002f60ea7c23c347e233c095b3a94c3f0691cf
Jul 30 20:23:59.527508 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:23:59.527473    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc"]
Jul 30 20:24:00.299053 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:24:00.298964    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v" event={"ID":"b390329e-625b-4236-a493-adb9ed8061cd","Type":"ContainerStarted","Data":"36e229fd5fc8bcc014ce0940d605a91a1543d6da19143970d910ebde6cb52c66"}
Jul 30 20:24:00.299229 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:24:00.299152    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v"
Jul 30 20:24:00.299229 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:24:00.299178    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v" event={"ID":"b390329e-625b-4236-a493-adb9ed8061cd","Type":"ContainerStarted","Data":"80b0efc69ed452aaf4204274bb0c7153ae8d4981340b930661c5d8263549938f"}
Jul 30 20:24:00.300504 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:24:00.300474    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc" event={"ID":"f3ffd11c-0d7b-4529-8edf-0925972813fd","Type":"ContainerStarted","Data":"8f0a9138744aff2aeca72dc05474e7b752ff1254498f3c24551461d41db99ff1"}
Jul 30 20:24:00.300660 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:24:00.300514    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc" event={"ID":"f3ffd11c-0d7b-4529-8edf-0925972813fd","Type":"ContainerStarted","Data":"8619b5c96e592168a9ab74bb7b002f60ea7c23c347e233c095b3a94c3f0691cf"}
Jul 30 20:24:00.300660 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:24:00.300592    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc"
Jul 30 20:24:00.320797 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:24:00.320744    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v" podStartSLOduration=1.9084420579999999 podStartE2EDuration="2.320728948s" totalImagesPullingTime="412.28689ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:23:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:24:00.315656852 +0000 UTC m=+3568.280237934" watchObservedRunningTime="2026-07-30 20:24:00.320728948 +0000 UTC m=+3568.285310030"
Jul 30 20:24:00.334045 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:24:00.333995    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc" podStartSLOduration=1.333981818 podStartE2EDuration="1.333981818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:23:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:24:00.331759968 +0000 UTC m=+3568.296341053" watchObservedRunningTime="2026-07-30 20:24:00.333981818 +0000 UTC m=+3568.298562918"
Jul 30 20:24:06.317686 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:24:06.317657    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v"
Jul 30 20:24:11.312980 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:24:11.312922    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc"
Jul 30 20:24:22.218109 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:24:22.218068    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:24:27.644411 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:24:27.644334    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:24:29.235629 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:24:29.235596    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc_f3ffd11c-0d7b-4529-8edf-0925972813fd/payload-processing/0.log"
Jul 30 20:24:32.892214 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:24:32.892184    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/1.log"
Jul 30 20:24:32.897045 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:24:32.897024    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 20:24:32.901807 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:24:32.901783    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/1.log"
Jul 30 20:24:32.906418 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:24:32.906399    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qnhsf_2b738474-0882-47d5-91ee-382257a723b3/ovn-acl-logging/0.log"
Jul 30 20:24:43.560251 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:24:43.560220    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:25:26.583850 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:26.583821    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc_f3ffd11c-0d7b-4529-8edf-0925972813fd/payload-processing/0.log"
Jul 30 20:25:26.853087 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:26.852999    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:25:27.150643 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.150558    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"]
Jul 30 20:25:27.154325 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.154303    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.156810 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.156791    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-23c617-openshift-default-dockercfg-9jn2v\""
Jul 30 20:25:27.166662 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.166636    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"]
Jul 30 20:25:27.169487 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.169451    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:25:27.224500 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.224471    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-credential-socket\") pod \"e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") " pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.224664 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.224504    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-workload-certs\") pod \"e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") " pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.224664 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.224542    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istio-data\") pod \"e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") " pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.224664 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.224597    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-workload-socket\") pod \"e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") " pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.224664 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.224618    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istio-envoy\") pod \"e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") " pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.224664 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.224634    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istio-podinfo\") pod \"e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") " pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.224664 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.224660    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istiod-ca-cert\") pod \"e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") " pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.225029 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.224747    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istio-token\") pod \"e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") " pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.225029 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.224790    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vcbn\" (UniqueName: \"kubernetes.io/projected/37fd3d5f-bef5-4326-91ed-aa74f6da022b-kube-api-access-6vcbn\") pod \"e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") " pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.326355 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.326319    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istio-token\") pod \"e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") " pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.326355 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.326360    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vcbn\" (UniqueName: \"kubernetes.io/projected/37fd3d5f-bef5-4326-91ed-aa74f6da022b-kube-api-access-6vcbn\") pod \"e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") " pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.326617 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.326383    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-credential-socket\") pod \"e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") " pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.326617 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.326404    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-workload-certs\") pod \"e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") " pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.326617 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.326439    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istio-data\") pod \"e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") " pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.326617 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.326488    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-workload-socket\") pod \"e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") " pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.326617 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.326518    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istio-envoy\") pod \"e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") " pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.326617 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.326545    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istio-podinfo\") pod \"e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") " pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.326617 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.326581    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istiod-ca-cert\") pod \"e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") " pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.327036 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.326857    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-credential-socket\") pod \"e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") " pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.327036 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.326892    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istio-data\") pod \"e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") " pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.327036 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.326932    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-workload-certs\") pod \"e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") " pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.327204 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.327108    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-workload-socket\") pod \"e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") " pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.327291 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.327273    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istiod-ca-cert\") pod \"e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") " pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.329232 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.329207    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istio-podinfo\") pod \"e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") " pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.329400 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.329379    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istio-envoy\") pod \"e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") " pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.346812 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.346784    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istio-token\") pod \"e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") " pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.346942 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.346823    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vcbn\" (UniqueName: \"kubernetes.io/projected/37fd3d5f-bef5-4326-91ed-aa74f6da022b-kube-api-access-6vcbn\") pod \"e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") " pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.348872 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.348846    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:25:27.465824 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.465792    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:27.610445 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.610416    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"]
Jul 30 20:25:27.612586 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:25:27.612558    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37fd3d5f_bef5_4326_91ed_aa74f6da022b.slice/crio-fab7fea99d2e7147b35fc89f096cdb2151d86d66a2efc6f26e97b7c051dcbbb0 WatchSource:0}: Error finding container fab7fea99d2e7147b35fc89f096cdb2151d86d66a2efc6f26e97b7c051dcbbb0: Status 404 returned error can't find the container with id fab7fea99d2e7147b35fc89f096cdb2151d86d66a2efc6f26e97b7c051dcbbb0
Jul 30 20:25:27.615232 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.615196    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 20:25:27.615355 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.615270    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 20:25:27.615355 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.615298    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 20:25:27.843120 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.843018    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg" event={"ID":"37fd3d5f-bef5-4326-91ed-aa74f6da022b","Type":"ContainerStarted","Data":"3db353d45ca8272fed1a40d7c2d983d6b861e9755a1dfb92d84a457261080dc7"}
Jul 30 20:25:27.843120 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.843089    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg" event={"ID":"37fd3d5f-bef5-4326-91ed-aa74f6da022b","Type":"ContainerStarted","Data":"fab7fea99d2e7147b35fc89f096cdb2151d86d66a2efc6f26e97b7c051dcbbb0"}
Jul 30 20:25:27.864213 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:27.864152    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg" podStartSLOduration=0.864136642 podStartE2EDuration="864.136642ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:25:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:25:27.861178324 +0000 UTC m=+3655.825759441" watchObservedRunningTime="2026-07-30 20:25:27.864136642 +0000 UTC m=+3655.828717723"
Jul 30 20:25:28.466647 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:28.466606    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:28.472375 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:28.472346    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:28.849277 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:28.849194    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:28.850339 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:28.850321    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:29.769378 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:29.769341    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:25:29.869689 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:29.869641    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:25:30.874175 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:30.874144    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg"]
Jul 30 20:25:30.877920 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:30.877902    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg"
Jul 30 20:25:30.880155 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:30.880128    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-23c617-serving-cert\""
Jul 30 20:25:30.888117 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:30.888091    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg"]
Jul 30 20:25:30.905657 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:30.905623    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:25:30.968721 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:30.968693    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db19d5e6-496c-4527-8559-7e05e4f092d8-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg\" (UID: \"db19d5e6-496c-4527-8559-7e05e4f092d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg"
Jul 30 20:25:30.968898 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:30.968793    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjdl5\" (UniqueName: \"kubernetes.io/projected/db19d5e6-496c-4527-8559-7e05e4f092d8-kube-api-access-hjdl5\") pod \"maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg\" (UID: \"db19d5e6-496c-4527-8559-7e05e4f092d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg"
Jul 30 20:25:30.968898 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:30.968815    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db19d5e6-496c-4527-8559-7e05e4f092d8-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg\" (UID: \"db19d5e6-496c-4527-8559-7e05e4f092d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg"
Jul 30 20:25:31.009693 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:31.009659    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:25:31.070295 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:31.070256    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjdl5\" (UniqueName: \"kubernetes.io/projected/db19d5e6-496c-4527-8559-7e05e4f092d8-kube-api-access-hjdl5\") pod \"maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg\" (UID: \"db19d5e6-496c-4527-8559-7e05e4f092d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg"
Jul 30 20:25:31.070499 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:31.070316    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db19d5e6-496c-4527-8559-7e05e4f092d8-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg\" (UID: \"db19d5e6-496c-4527-8559-7e05e4f092d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg"
Jul 30 20:25:31.070499 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:31.070423    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db19d5e6-496c-4527-8559-7e05e4f092d8-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg\" (UID: \"db19d5e6-496c-4527-8559-7e05e4f092d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg"
Jul 30 20:25:31.071177 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:31.071151    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db19d5e6-496c-4527-8559-7e05e4f092d8-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg\" (UID: \"db19d5e6-496c-4527-8559-7e05e4f092d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg"
Jul 30 20:25:31.072834 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:31.072809    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db19d5e6-496c-4527-8559-7e05e4f092d8-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg\" (UID: \"db19d5e6-496c-4527-8559-7e05e4f092d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg"
Jul 30 20:25:31.079299 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:31.079275    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjdl5\" (UniqueName: \"kubernetes.io/projected/db19d5e6-496c-4527-8559-7e05e4f092d8-kube-api-access-hjdl5\") pod \"maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg\" (UID: \"db19d5e6-496c-4527-8559-7e05e4f092d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg"
Jul 30 20:25:31.191559 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:31.191463    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg"
Jul 30 20:25:31.340300 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:25:31.340252    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb19d5e6_496c_4527_8559_7e05e4f092d8.slice/crio-0de06247cf33821513c3d7e98b1cf7fb96fe5b4c05b0c574cce5217d86084bbf WatchSource:0}: Error finding container 0de06247cf33821513c3d7e98b1cf7fb96fe5b4c05b0c574cce5217d86084bbf: Status 404 returned error can't find the container with id 0de06247cf33821513c3d7e98b1cf7fb96fe5b4c05b0c574cce5217d86084bbf
Jul 30 20:25:31.340532 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:31.340498    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg"]
Jul 30 20:25:31.422455 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:31.422424    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:25:31.523510 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:31.523453    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:25:31.868179 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:31.868136    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg" event={"ID":"db19d5e6-496c-4527-8559-7e05e4f092d8","Type":"ContainerStarted","Data":"16e510881b226d500008173b631a24d708dc14de4e90a223d3effdc0149185d3"}
Jul 30 20:25:31.868346 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:31.868209    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg"
Jul 30 20:25:31.868346 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:31.868240    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg" event={"ID":"db19d5e6-496c-4527-8559-7e05e4f092d8","Type":"ContainerStarted","Data":"0de06247cf33821513c3d7e98b1cf7fb96fe5b4c05b0c574cce5217d86084bbf"}
Jul 30 20:25:31.886474 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:31.886420    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg" podStartSLOduration=1.481571825 podStartE2EDuration="1.886405657s" totalImagesPullingTime="404.833832ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:25:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:25:31.883606883 +0000 UTC m=+3659.848187964" watchObservedRunningTime="2026-07-30 20:25:31.886405657 +0000 UTC m=+3659.850986761"
Jul 30 20:25:37.880820 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:37.880787    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg"
Jul 30 20:25:48.983754 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:48.983715    2554 generic.go:356] "Generic (PLEG): container finished" podID="37fd3d5f-bef5-4326-91ed-aa74f6da022b" containerID="3db353d45ca8272fed1a40d7c2d983d6b861e9755a1dfb92d84a457261080dc7" exitCode=0
Jul 30 20:25:48.984230 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:48.983789    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg" event={"ID":"37fd3d5f-bef5-4326-91ed-aa74f6da022b","Type":"ContainerDied","Data":"3db353d45ca8272fed1a40d7c2d983d6b861e9755a1dfb92d84a457261080dc7"}
Jul 30 20:25:48.984276 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:48.984255    2554 scope.go:117] "RemoveContainer" containerID="3db353d45ca8272fed1a40d7c2d983d6b861e9755a1dfb92d84a457261080dc7"
Jul 30 20:25:48.985467 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:48.985438    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 20:25:48.985520 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:48.985505    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 20:25:48.985556 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:48.985539    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 30 20:25:49.991740 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:49.991693    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg" event={"ID":"37fd3d5f-bef5-4326-91ed-aa74f6da022b","Type":"ContainerStarted","Data":"a0fce1afab10a801cced8eb95a01b13ba879154b9a63f2fc8e9991d939e4a4c7"}
Jul 30 20:25:50.466490 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:50.466457    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:50.466707 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:50.466677    2554 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Jul 30 20:25:50.466790 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:50.466755    2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg" podUID="37fd3d5f-bef5-4326-91ed-aa74f6da022b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Jul 30 20:25:51.466570 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:51.466535    2554 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Jul 30 20:25:51.467000 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:51.466593    2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg" podUID="37fd3d5f-bef5-4326-91ed-aa74f6da022b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Jul 30 20:25:52.466877 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:52.466844    2554 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Jul 30 20:25:52.467369 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:52.466898    2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg" podUID="37fd3d5f-bef5-4326-91ed-aa74f6da022b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Jul 30 20:25:53.466455 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:53.466407    2554 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Jul 30 20:25:53.466672 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:53.466495    2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg" podUID="37fd3d5f-bef5-4326-91ed-aa74f6da022b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Jul 30 20:25:54.469977 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:54.469884    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:54.470400 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:54.470196    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:54.470838 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:54.470816    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:25:54.764331 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:54.764238    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:25:55.466647 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:55.466606    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23c617-gcz5x"]
Jul 30 20:25:55.469627 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:55.469605    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23c617-gcz5x"
Jul 30 20:25:55.472339 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:55.472320    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7t7kk\""
Jul 30 20:25:55.478133 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:55.478111    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23c617-gcz5x"]
Jul 30 20:25:55.545152 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:55.545107    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aefa49d6-1c86-421f-9785-441bbb42aacc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-23c617-gcz5x\" (UID: \"aefa49d6-1c86-421f-9785-441bbb42aacc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23c617-gcz5x"
Jul 30 20:25:55.645736 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:55.645703    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aefa49d6-1c86-421f-9785-441bbb42aacc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-23c617-gcz5x\" (UID: \"aefa49d6-1c86-421f-9785-441bbb42aacc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23c617-gcz5x"
Jul 30 20:25:55.646348 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:55.646327    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aefa49d6-1c86-421f-9785-441bbb42aacc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-23c617-gcz5x\" (UID: \"aefa49d6-1c86-421f-9785-441bbb42aacc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23c617-gcz5x"
Jul 30 20:25:55.782332 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:55.782253    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23c617-gcz5x"
Jul 30 20:25:55.926621 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:55.926589    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23c617-gcz5x"]
Jul 30 20:25:55.928623 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:25:55.928597    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaefa49d6_1c86_421f_9785_441bbb42aacc.slice/crio-d822cd48cd7cee466b69bf02afc8690fc680423a1dde8583a073820267af2e30 WatchSource:0}: Error finding container d822cd48cd7cee466b69bf02afc8690fc680423a1dde8583a073820267af2e30: Status 404 returned error can't find the container with id d822cd48cd7cee466b69bf02afc8690fc680423a1dde8583a073820267af2e30
Jul 30 20:25:56.032059 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:56.032018    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23c617-gcz5x" event={"ID":"aefa49d6-1c86-421f-9785-441bbb42aacc","Type":"ContainerStarted","Data":"d822cd48cd7cee466b69bf02afc8690fc680423a1dde8583a073820267af2e30"}
Jul 30 20:25:57.040495 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:57.040455    2554 generic.go:356] "Generic (PLEG): container finished" podID="aefa49d6-1c86-421f-9785-441bbb42aacc" containerID="5d0a1ad343a20adbc4b93a94791ec19a5d999b5ab7dbd670d3755d24c618c874" exitCode=0
Jul 30 20:25:57.040886 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:57.040541    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23c617-gcz5x" event={"ID":"aefa49d6-1c86-421f-9785-441bbb42aacc","Type":"ContainerDied","Data":"5d0a1ad343a20adbc4b93a94791ec19a5d999b5ab7dbd670d3755d24c618c874"}
Jul 30 20:25:58.191388 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:58.191356    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23c617-gcz5x"
Jul 30 20:25:58.272500 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:58.272464    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aefa49d6-1c86-421f-9785-441bbb42aacc-maas-api-service-ca\") pod \"aefa49d6-1c86-421f-9785-441bbb42aacc\" (UID: \"aefa49d6-1c86-421f-9785-441bbb42aacc\") "
Jul 30 20:25:58.272799 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:58.272775    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aefa49d6-1c86-421f-9785-441bbb42aacc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "aefa49d6-1c86-421f-9785-441bbb42aacc" (UID: "aefa49d6-1c86-421f-9785-441bbb42aacc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:25:58.374241 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:58.374158    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aefa49d6-1c86-421f-9785-441bbb42aacc-maas-api-service-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:25:59.055266 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:59.055235    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23c617-gcz5x"
Jul 30 20:25:59.055437 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:59.055268    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23c617-gcz5x" event={"ID":"aefa49d6-1c86-421f-9785-441bbb42aacc","Type":"ContainerDied","Data":"d822cd48cd7cee466b69bf02afc8690fc680423a1dde8583a073820267af2e30"}
Jul 30 20:25:59.055437 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:25:59.055317    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d822cd48cd7cee466b69bf02afc8690fc680423a1dde8583a073820267af2e30"
Jul 30 20:26:01.386402 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:01.386360    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg"]
Jul 30 20:26:01.386886 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:01.386642    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg" podUID="db19d5e6-496c-4527-8559-7e05e4f092d8" containerName="maas-api" containerID="cri-o://16e510881b226d500008173b631a24d708dc14de4e90a223d3effdc0149185d3" gracePeriod=30
Jul 30 20:26:01.410701 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:01.410652    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:26:01.498034 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:01.497995    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:26:01.554127 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:01.554076    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:26:01.590019 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:01.589975    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23c617-gcz5x"]
Jul 30 20:26:01.597045 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:01.597012    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-23c617-gcz5x"]
Jul 30 20:26:01.660277 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:01.660252    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg"
Jul 30 20:26:01.697464 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:01.697428    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:26:01.704772 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:01.704744    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db19d5e6-496c-4527-8559-7e05e4f092d8-openshift-service-ca\") pod \"db19d5e6-496c-4527-8559-7e05e4f092d8\" (UID: \"db19d5e6-496c-4527-8559-7e05e4f092d8\") "
Jul 30 20:26:01.704905 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:01.704788    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjdl5\" (UniqueName: \"kubernetes.io/projected/db19d5e6-496c-4527-8559-7e05e4f092d8-kube-api-access-hjdl5\") pod \"db19d5e6-496c-4527-8559-7e05e4f092d8\" (UID: \"db19d5e6-496c-4527-8559-7e05e4f092d8\") "
Jul 30 20:26:01.704905 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:01.704871    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db19d5e6-496c-4527-8559-7e05e4f092d8-maas-api-tls\") pod \"db19d5e6-496c-4527-8559-7e05e4f092d8\" (UID: \"db19d5e6-496c-4527-8559-7e05e4f092d8\") "
Jul 30 20:26:01.705193 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:01.705158    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db19d5e6-496c-4527-8559-7e05e4f092d8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "db19d5e6-496c-4527-8559-7e05e4f092d8" (UID: "db19d5e6-496c-4527-8559-7e05e4f092d8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:26:01.707470 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:01.707443    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db19d5e6-496c-4527-8559-7e05e4f092d8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "db19d5e6-496c-4527-8559-7e05e4f092d8" (UID: "db19d5e6-496c-4527-8559-7e05e4f092d8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:26:01.707594 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:01.707472    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db19d5e6-496c-4527-8559-7e05e4f092d8-kube-api-access-hjdl5" (OuterVolumeSpecName: "kube-api-access-hjdl5") pod "db19d5e6-496c-4527-8559-7e05e4f092d8" (UID: "db19d5e6-496c-4527-8559-7e05e4f092d8"). InnerVolumeSpecName "kube-api-access-hjdl5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:26:01.806710 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:01.806673    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db19d5e6-496c-4527-8559-7e05e4f092d8-openshift-service-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:26:01.806710 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:01.806705    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjdl5\" (UniqueName: \"kubernetes.io/projected/db19d5e6-496c-4527-8559-7e05e4f092d8-kube-api-access-hjdl5\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:26:01.806710 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:01.806716    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db19d5e6-496c-4527-8559-7e05e4f092d8-maas-api-tls\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:26:01.957823 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:01.957740    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:26:02.074780 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:02.074742    2554 generic.go:356] "Generic (PLEG): container finished" podID="db19d5e6-496c-4527-8559-7e05e4f092d8" containerID="16e510881b226d500008173b631a24d708dc14de4e90a223d3effdc0149185d3" exitCode=0
Jul 30 20:26:02.074977 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:02.074790    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg"
Jul 30 20:26:02.074977 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:02.074826    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg" event={"ID":"db19d5e6-496c-4527-8559-7e05e4f092d8","Type":"ContainerDied","Data":"16e510881b226d500008173b631a24d708dc14de4e90a223d3effdc0149185d3"}
Jul 30 20:26:02.074977 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:02.074883    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg" event={"ID":"db19d5e6-496c-4527-8559-7e05e4f092d8","Type":"ContainerDied","Data":"0de06247cf33821513c3d7e98b1cf7fb96fe5b4c05b0c574cce5217d86084bbf"}
Jul 30 20:26:02.074977 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:02.074907    2554 scope.go:117] "RemoveContainer" containerID="16e510881b226d500008173b631a24d708dc14de4e90a223d3effdc0149185d3"
Jul 30 20:26:02.085348 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:02.085330    2554 scope.go:117] "RemoveContainer" containerID="16e510881b226d500008173b631a24d708dc14de4e90a223d3effdc0149185d3"
Jul 30 20:26:02.085615 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:26:02.085597    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16e510881b226d500008173b631a24d708dc14de4e90a223d3effdc0149185d3\": container with ID starting with 16e510881b226d500008173b631a24d708dc14de4e90a223d3effdc0149185d3 not found: ID does not exist" containerID="16e510881b226d500008173b631a24d708dc14de4e90a223d3effdc0149185d3"
Jul 30 20:26:02.085680 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:02.085620    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16e510881b226d500008173b631a24d708dc14de4e90a223d3effdc0149185d3"} err="failed to get container status \"16e510881b226d500008173b631a24d708dc14de4e90a223d3effdc0149185d3\": rpc error: code = NotFound desc = could not find container \"16e510881b226d500008173b631a24d708dc14de4e90a223d3effdc0149185d3\": container with ID starting with 16e510881b226d500008173b631a24d708dc14de4e90a223d3effdc0149185d3 not found: ID does not exist"
Jul 30 20:26:02.097495 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:02.097468    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg"]
Jul 30 20:26:02.100297 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:02.100193    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-23c617-85b8474f9c-ws4hg"]
Jul 30 20:26:02.495630 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:02.495594    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aefa49d6-1c86-421f-9785-441bbb42aacc" path="/var/lib/kubelet/pods/aefa49d6-1c86-421f-9785-441bbb42aacc/volumes"
Jul 30 20:26:02.496084 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:02.496067    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db19d5e6-496c-4527-8559-7e05e4f092d8" path="/var/lib/kubelet/pods/db19d5e6-496c-4527-8559-7e05e4f092d8/volumes"
Jul 30 20:26:07.545257 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:07.545211    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:26:07.646826 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:07.646773    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:26:07.797649 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:07.797563    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"]
Jul 30 20:26:07.797788 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:07.797763    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg" podUID="37fd3d5f-bef5-4326-91ed-aa74f6da022b" containerName="istio-proxy" containerID="cri-o://a0fce1afab10a801cced8eb95a01b13ba879154b9a63f2fc8e9991d939e4a4c7" gracePeriod=30
Jul 30 20:26:07.812336 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:07.812294    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:26:08.039178 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:08.039142    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:26:09.473267 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:09.473225    2554 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 30 20:26:09.473644 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:09.473296    2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg" podUID="37fd3d5f-bef5-4326-91ed-aa74f6da022b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 20:26:11.416082 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:11.416042    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0c599c-c2p6k"]
Jul 30 20:26:11.416703 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:11.416683    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aefa49d6-1c86-421f-9785-441bbb42aacc" containerName="revoke-keys"
Jul 30 20:26:11.416800 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:11.416707    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="aefa49d6-1c86-421f-9785-441bbb42aacc" containerName="revoke-keys"
Jul 30 20:26:11.416800 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:11.416736    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db19d5e6-496c-4527-8559-7e05e4f092d8" containerName="maas-api"
Jul 30 20:26:11.416800 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:11.416745    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="db19d5e6-496c-4527-8559-7e05e4f092d8" containerName="maas-api"
Jul 30 20:26:11.416979 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:11.416853    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="db19d5e6-496c-4527-8559-7e05e4f092d8" containerName="maas-api"
Jul 30 20:26:11.416979 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:11.416870    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="aefa49d6-1c86-421f-9785-441bbb42aacc" containerName="revoke-keys"
Jul 30 20:26:11.420621 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:11.420599    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0c599c-c2p6k"
Jul 30 20:26:11.423465 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:11.423310    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7t7kk\""
Jul 30 20:26:11.426427 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:11.426406    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0c599c-c2p6k"]
Jul 30 20:26:11.501014 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:11.500947    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56a9f067-25f1-4a72-8edd-8cb0d07e8234-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-0c599c-c2p6k\" (UID: \"56a9f067-25f1-4a72-8edd-8cb0d07e8234\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0c599c-c2p6k"
Jul 30 20:26:11.602209 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:11.602165    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56a9f067-25f1-4a72-8edd-8cb0d07e8234-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-0c599c-c2p6k\" (UID: \"56a9f067-25f1-4a72-8edd-8cb0d07e8234\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0c599c-c2p6k"
Jul 30 20:26:11.603041 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:11.603018    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56a9f067-25f1-4a72-8edd-8cb0d07e8234-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-0c599c-c2p6k\" (UID: \"56a9f067-25f1-4a72-8edd-8cb0d07e8234\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0c599c-c2p6k"
Jul 30 20:26:11.734932 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:11.734897    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0c599c-c2p6k"
Jul 30 20:26:11.870772 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:11.870746    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0c599c-c2p6k"]
Jul 30 20:26:11.871650 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:26:11.871626    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56a9f067_25f1_4a72_8edd_8cb0d07e8234.slice/crio-85cced31926fabe48cf583b4f5b55780a11cc0ab7cb2b8dcd312d001618494e8 WatchSource:0}: Error finding container 85cced31926fabe48cf583b4f5b55780a11cc0ab7cb2b8dcd312d001618494e8: Status 404 returned error can't find the container with id 85cced31926fabe48cf583b4f5b55780a11cc0ab7cb2b8dcd312d001618494e8
Jul 30 20:26:12.145821 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:12.145784    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0c599c-c2p6k" event={"ID":"56a9f067-25f1-4a72-8edd-8cb0d07e8234","Type":"ContainerStarted","Data":"85cced31926fabe48cf583b4f5b55780a11cc0ab7cb2b8dcd312d001618494e8"}
Jul 30 20:26:13.057251 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.057226    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:26:13.116745 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.116706    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-workload-socket\") pod \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") "
Jul 30 20:26:13.116935 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.116768    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vcbn\" (UniqueName: \"kubernetes.io/projected/37fd3d5f-bef5-4326-91ed-aa74f6da022b-kube-api-access-6vcbn\") pod \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") "
Jul 30 20:26:13.116935 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.116828    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istio-data\") pod \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") "
Jul 30 20:26:13.116935 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.116914    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-credential-socket\") pod \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") "
Jul 30 20:26:13.117252 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.116944    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istio-podinfo\") pod \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") "
Jul 30 20:26:13.117252 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.116998    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istio-token\") pod \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") "
Jul 30 20:26:13.117252 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.117022    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istiod-ca-cert\") pod \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") "
Jul 30 20:26:13.117252 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.117038    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istio-envoy\") pod \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") "
Jul 30 20:26:13.117252 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.117076    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-workload-certs\") pod \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\" (UID: \"37fd3d5f-bef5-4326-91ed-aa74f6da022b\") "
Jul 30 20:26:13.117252 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.117128    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "37fd3d5f-bef5-4326-91ed-aa74f6da022b" (UID: "37fd3d5f-bef5-4326-91ed-aa74f6da022b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:26:13.117564 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.117415    2554 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-workload-socket\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:26:13.117642 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.117617    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "37fd3d5f-bef5-4326-91ed-aa74f6da022b" (UID: "37fd3d5f-bef5-4326-91ed-aa74f6da022b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:26:13.117709 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.117649    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "37fd3d5f-bef5-4326-91ed-aa74f6da022b" (UID: "37fd3d5f-bef5-4326-91ed-aa74f6da022b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:26:13.118117 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.118010    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "37fd3d5f-bef5-4326-91ed-aa74f6da022b" (UID: "37fd3d5f-bef5-4326-91ed-aa74f6da022b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:26:13.118117 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.118091    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istio-data" (OuterVolumeSpecName: "istio-data") pod "37fd3d5f-bef5-4326-91ed-aa74f6da022b" (UID: "37fd3d5f-bef5-4326-91ed-aa74f6da022b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:26:13.119302 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.119251    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37fd3d5f-bef5-4326-91ed-aa74f6da022b-kube-api-access-6vcbn" (OuterVolumeSpecName: "kube-api-access-6vcbn") pod "37fd3d5f-bef5-4326-91ed-aa74f6da022b" (UID: "37fd3d5f-bef5-4326-91ed-aa74f6da022b"). InnerVolumeSpecName "kube-api-access-6vcbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:26:13.119820 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.119791    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "37fd3d5f-bef5-4326-91ed-aa74f6da022b" (UID: "37fd3d5f-bef5-4326-91ed-aa74f6da022b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:26:13.119978 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.119936    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istio-token" (OuterVolumeSpecName: "istio-token") pod "37fd3d5f-bef5-4326-91ed-aa74f6da022b" (UID: "37fd3d5f-bef5-4326-91ed-aa74f6da022b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:26:13.120055 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.120007    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "37fd3d5f-bef5-4326-91ed-aa74f6da022b" (UID: "37fd3d5f-bef5-4326-91ed-aa74f6da022b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 20:26:13.153540 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.153507    2554 generic.go:356] "Generic (PLEG): container finished" podID="56a9f067-25f1-4a72-8edd-8cb0d07e8234" containerID="2919c701e403b8e0b85fcfe86249beb7328422749b8a2d7515aeef5635c356fb" exitCode=0
Jul 30 20:26:13.153689 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.153594    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0c599c-c2p6k" event={"ID":"56a9f067-25f1-4a72-8edd-8cb0d07e8234","Type":"ContainerDied","Data":"2919c701e403b8e0b85fcfe86249beb7328422749b8a2d7515aeef5635c356fb"}
Jul 30 20:26:13.155477 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.155451    2554 generic.go:356] "Generic (PLEG): container finished" podID="37fd3d5f-bef5-4326-91ed-aa74f6da022b" containerID="a0fce1afab10a801cced8eb95a01b13ba879154b9a63f2fc8e9991d939e4a4c7" exitCode=0
Jul 30 20:26:13.155588 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.155503    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"
Jul 30 20:26:13.155588 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.155531    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg" event={"ID":"37fd3d5f-bef5-4326-91ed-aa74f6da022b","Type":"ContainerDied","Data":"a0fce1afab10a801cced8eb95a01b13ba879154b9a63f2fc8e9991d939e4a4c7"}
Jul 30 20:26:13.155588 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.155580    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg" event={"ID":"37fd3d5f-bef5-4326-91ed-aa74f6da022b","Type":"ContainerDied","Data":"fab7fea99d2e7147b35fc89f096cdb2151d86d66a2efc6f26e97b7c051dcbbb0"}
Jul 30 20:26:13.155701 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.155604    2554 scope.go:117] "RemoveContainer" containerID="a0fce1afab10a801cced8eb95a01b13ba879154b9a63f2fc8e9991d939e4a4c7"
Jul 30 20:26:13.166213 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.166197    2554 scope.go:117] "RemoveContainer" containerID="3db353d45ca8272fed1a40d7c2d983d6b861e9755a1dfb92d84a457261080dc7"
Jul 30 20:26:13.175618 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.175601    2554 scope.go:117] "RemoveContainer" containerID="a0fce1afab10a801cced8eb95a01b13ba879154b9a63f2fc8e9991d939e4a4c7"
Jul 30 20:26:13.175883 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:26:13.175867    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0fce1afab10a801cced8eb95a01b13ba879154b9a63f2fc8e9991d939e4a4c7\": container with ID starting with a0fce1afab10a801cced8eb95a01b13ba879154b9a63f2fc8e9991d939e4a4c7 not found: ID does not exist" containerID="a0fce1afab10a801cced8eb95a01b13ba879154b9a63f2fc8e9991d939e4a4c7"
Jul 30 20:26:13.175927 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.175892    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0fce1afab10a801cced8eb95a01b13ba879154b9a63f2fc8e9991d939e4a4c7"} err="failed to get container status \"a0fce1afab10a801cced8eb95a01b13ba879154b9a63f2fc8e9991d939e4a4c7\": rpc error: code = NotFound desc = could not find container \"a0fce1afab10a801cced8eb95a01b13ba879154b9a63f2fc8e9991d939e4a4c7\": container with ID starting with a0fce1afab10a801cced8eb95a01b13ba879154b9a63f2fc8e9991d939e4a4c7 not found: ID does not exist"
Jul 30 20:26:13.175927 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.175907    2554 scope.go:117] "RemoveContainer" containerID="3db353d45ca8272fed1a40d7c2d983d6b861e9755a1dfb92d84a457261080dc7"
Jul 30 20:26:13.176146 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:26:13.176131    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3db353d45ca8272fed1a40d7c2d983d6b861e9755a1dfb92d84a457261080dc7\": container with ID starting with 3db353d45ca8272fed1a40d7c2d983d6b861e9755a1dfb92d84a457261080dc7 not found: ID does not exist" containerID="3db353d45ca8272fed1a40d7c2d983d6b861e9755a1dfb92d84a457261080dc7"
Jul 30 20:26:13.176196 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.176150    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3db353d45ca8272fed1a40d7c2d983d6b861e9755a1dfb92d84a457261080dc7"} err="failed to get container status \"3db353d45ca8272fed1a40d7c2d983d6b861e9755a1dfb92d84a457261080dc7\": rpc error: code = NotFound desc = could not find container \"3db353d45ca8272fed1a40d7c2d983d6b861e9755a1dfb92d84a457261080dc7\": container with ID starting with 3db353d45ca8272fed1a40d7c2d983d6b861e9755a1dfb92d84a457261080dc7 not found: ID does not exist"
Jul 30 20:26:13.191571 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.191542    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"]
Jul 30 20:26:13.193431 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.193407    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-23c617-openshift-default-896cbd5b8-4bzgg"]
Jul 30 20:26:13.218028 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.217996    2554 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-credential-socket\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:26:13.218028 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.218030    2554 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istio-podinfo\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:26:13.218170 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.218041    2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istio-token\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:26:13.218170 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.218049    2554 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istiod-ca-cert\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:26:13.218170 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.218057    2554 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istio-envoy\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:26:13.218170 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.218064    2554 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-workload-certs\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:26:13.218170 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.218072    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vcbn\" (UniqueName: \"kubernetes.io/projected/37fd3d5f-bef5-4326-91ed-aa74f6da022b-kube-api-access-6vcbn\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:26:13.218170 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:13.218081    2554 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37fd3d5f-bef5-4326-91ed-aa74f6da022b-istio-data\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:26:14.311310 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:14.311281    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0c599c-c2p6k"
Jul 30 20:26:14.429187 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:14.429094    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56a9f067-25f1-4a72-8edd-8cb0d07e8234-maas-api-service-ca\") pod \"56a9f067-25f1-4a72-8edd-8cb0d07e8234\" (UID: \"56a9f067-25f1-4a72-8edd-8cb0d07e8234\") "
Jul 30 20:26:14.429389 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:14.429364    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56a9f067-25f1-4a72-8edd-8cb0d07e8234-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "56a9f067-25f1-4a72-8edd-8cb0d07e8234" (UID: "56a9f067-25f1-4a72-8edd-8cb0d07e8234"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:26:14.487441 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:14.487408    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37fd3d5f-bef5-4326-91ed-aa74f6da022b" path="/var/lib/kubelet/pods/37fd3d5f-bef5-4326-91ed-aa74f6da022b/volumes"
Jul 30 20:26:14.530523 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:14.530484    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56a9f067-25f1-4a72-8edd-8cb0d07e8234-maas-api-service-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:26:15.170661 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:15.170632    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0c599c-c2p6k"
Jul 30 20:26:15.170813 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:15.170670    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0c599c-c2p6k" event={"ID":"56a9f067-25f1-4a72-8edd-8cb0d07e8234","Type":"ContainerDied","Data":"85cced31926fabe48cf583b4f5b55780a11cc0ab7cb2b8dcd312d001618494e8"}
Jul 30 20:26:15.170813 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:15.170722    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="85cced31926fabe48cf583b4f5b55780a11cc0ab7cb2b8dcd312d001618494e8"
Jul 30 20:26:27.048687 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:27.048632    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:26:27.148885 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:27.148831    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:26:27.203830 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:27.203801    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0c599c-c2p6k"]
Jul 30 20:26:27.204818 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:27.204792    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:26:27.206806 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:27.206785    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0c599c-c2p6k"]
Jul 30 20:26:27.597506 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:27.597469    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:26:28.487566 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:28.487528    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56a9f067-25f1-4a72-8edd-8cb0d07e8234" path="/var/lib/kubelet/pods/56a9f067-25f1-4a72-8edd-8cb0d07e8234/volumes"
Jul 30 20:26:30.952257 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:30.952206    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:26:31.053604 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:31.053554    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:26:32.692200 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:32.692159    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:26:32.793333 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:26:32.793295    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:27:09.418453 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:09.418382    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:27:09.565087 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:09.565052    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:27:15.198864 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:15.198802    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:27:29.525761 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:29.525660    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:27:29.620518 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:29.620477    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:27:29.787381 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:29.787290    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:27:30.032340 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:30.032303    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:27:30.140075 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:30.139990    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d67d52-pp7bj"]
Jul 30 20:27:30.140421 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:30.140408    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37fd3d5f-bef5-4326-91ed-aa74f6da022b" containerName="istio-proxy"
Jul 30 20:27:30.140472 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:30.140422    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="37fd3d5f-bef5-4326-91ed-aa74f6da022b" containerName="istio-proxy"
Jul 30 20:27:30.140472 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:30.140431    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56a9f067-25f1-4a72-8edd-8cb0d07e8234" containerName="revoke-keys"
Jul 30 20:27:30.140472 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:30.140437    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="56a9f067-25f1-4a72-8edd-8cb0d07e8234" containerName="revoke-keys"
Jul 30 20:27:30.140472 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:30.140461    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37fd3d5f-bef5-4326-91ed-aa74f6da022b" containerName="istio-proxy"
Jul 30 20:27:30.140472 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:30.140467    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="37fd3d5f-bef5-4326-91ed-aa74f6da022b" containerName="istio-proxy"
Jul 30 20:27:30.140641 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:30.140545    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="56a9f067-25f1-4a72-8edd-8cb0d07e8234" containerName="revoke-keys"
Jul 30 20:27:30.140641 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:30.140554    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="37fd3d5f-bef5-4326-91ed-aa74f6da022b" containerName="istio-proxy"
Jul 30 20:27:30.140641 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:30.140560    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="37fd3d5f-bef5-4326-91ed-aa74f6da022b" containerName="istio-proxy"
Jul 30 20:27:30.143861 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:30.143837    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d67d52-pp7bj"
Jul 30 20:27:30.146288 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:30.146265    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7t7kk\""
Jul 30 20:27:30.152040 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:30.152015    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d67d52-pp7bj"]
Jul 30 20:27:30.268975 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:30.268922    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80fed8d0-d0d6-47b5-8765-cc6dad641f14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-d67d52-pp7bj\" (UID: \"80fed8d0-d0d6-47b5-8765-cc6dad641f14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d67d52-pp7bj"
Jul 30 20:27:30.369764 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:30.369732    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80fed8d0-d0d6-47b5-8765-cc6dad641f14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-d67d52-pp7bj\" (UID: \"80fed8d0-d0d6-47b5-8765-cc6dad641f14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d67d52-pp7bj"
Jul 30 20:27:30.370345 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:30.370326    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80fed8d0-d0d6-47b5-8765-cc6dad641f14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-d67d52-pp7bj\" (UID: \"80fed8d0-d0d6-47b5-8765-cc6dad641f14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d67d52-pp7bj"
Jul 30 20:27:30.458013 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:30.457899    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d67d52-pp7bj"
Jul 30 20:27:30.604901 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:27:30.604873    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80fed8d0_d0d6_47b5_8765_cc6dad641f14.slice/crio-538e64850dc1b3b90f74f8dcc5ad183f9e197a24dd1900535d4b0cef3e56cb15 WatchSource:0}: Error finding container 538e64850dc1b3b90f74f8dcc5ad183f9e197a24dd1900535d4b0cef3e56cb15: Status 404 returned error can't find the container with id 538e64850dc1b3b90f74f8dcc5ad183f9e197a24dd1900535d4b0cef3e56cb15
Jul 30 20:27:30.605532 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:30.605504    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d67d52-pp7bj"]
Jul 30 20:27:30.639585 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:30.639556    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d67d52-pp7bj" event={"ID":"80fed8d0-d0d6-47b5-8765-cc6dad641f14","Type":"ContainerStarted","Data":"538e64850dc1b3b90f74f8dcc5ad183f9e197a24dd1900535d4b0cef3e56cb15"}
Jul 30 20:27:31.647653 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:31.647564    2554 generic.go:356] "Generic (PLEG): container finished" podID="80fed8d0-d0d6-47b5-8765-cc6dad641f14" containerID="b31816fb96becc23c6248b0b25b58a95431e29a1311b5c46e0e7cc9fedcff32c" exitCode=0
Jul 30 20:27:31.648058 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:31.647650    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d67d52-pp7bj" event={"ID":"80fed8d0-d0d6-47b5-8765-cc6dad641f14","Type":"ContainerDied","Data":"b31816fb96becc23c6248b0b25b58a95431e29a1311b5c46e0e7cc9fedcff32c"}
Jul 30 20:27:32.795820 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:32.795793    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d67d52-pp7bj"
Jul 30 20:27:32.897027 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:32.896997    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80fed8d0-d0d6-47b5-8765-cc6dad641f14-maas-api-service-ca\") pod \"80fed8d0-d0d6-47b5-8765-cc6dad641f14\" (UID: \"80fed8d0-d0d6-47b5-8765-cc6dad641f14\") "
Jul 30 20:27:32.897319 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:32.897297    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80fed8d0-d0d6-47b5-8765-cc6dad641f14-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "80fed8d0-d0d6-47b5-8765-cc6dad641f14" (UID: "80fed8d0-d0d6-47b5-8765-cc6dad641f14"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:27:32.998221 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:32.998183    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/80fed8d0-d0d6-47b5-8765-cc6dad641f14-maas-api-service-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:27:33.661079 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:33.661053    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d67d52-pp7bj"
Jul 30 20:27:33.661264 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:33.661045    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d67d52-pp7bj" event={"ID":"80fed8d0-d0d6-47b5-8765-cc6dad641f14","Type":"ContainerDied","Data":"538e64850dc1b3b90f74f8dcc5ad183f9e197a24dd1900535d4b0cef3e56cb15"}
Jul 30 20:27:33.661264 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:33.661203    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="538e64850dc1b3b90f74f8dcc5ad183f9e197a24dd1900535d4b0cef3e56cb15"
Jul 30 20:27:40.934907 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:40.934863    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v"]
Jul 30 20:27:40.935435 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:40.935106    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v" podUID="b390329e-625b-4236-a493-adb9ed8061cd" containerName="maas-api" containerID="cri-o://36e229fd5fc8bcc014ce0940d605a91a1543d6da19143970d910ebde6cb52c66" gracePeriod=30
Jul 30 20:27:40.957680 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:40.957647    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:27:40.991480 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:40.991446    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc"]
Jul 30 20:27:40.991738 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:40.991701    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc" podUID="f3ffd11c-0d7b-4529-8edf-0925972813fd" containerName="payload-processing" containerID="cri-o://8f0a9138744aff2aeca72dc05474e7b752ff1254498f3c24551461d41db99ff1" gracePeriod=30
Jul 30 20:27:41.023059 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.023030    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:27:41.121296 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.121265    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:27:41.135882 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.135837    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d67d52-pp7bj"]
Jul 30 20:27:41.138667 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.138641    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d67d52-pp7bj"]
Jul 30 20:27:41.221881 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.221848    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:27:41.259498 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.259474    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v"
Jul 30 20:27:41.284467 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.284444    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc"
Jul 30 20:27:41.386825 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.386789    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-998jp\" (UniqueName: \"kubernetes.io/projected/b390329e-625b-4236-a493-adb9ed8061cd-kube-api-access-998jp\") pod \"b390329e-625b-4236-a493-adb9ed8061cd\" (UID: \"b390329e-625b-4236-a493-adb9ed8061cd\") "
Jul 30 20:27:41.387023 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.386838    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3ffd11c-0d7b-4529-8edf-0925972813fd-plugins-config-volume\") pod \"f3ffd11c-0d7b-4529-8edf-0925972813fd\" (UID: \"f3ffd11c-0d7b-4529-8edf-0925972813fd\") "
Jul 30 20:27:41.387023 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.386909    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b390329e-625b-4236-a493-adb9ed8061cd-maas-api-tls\") pod \"b390329e-625b-4236-a493-adb9ed8061cd\" (UID: \"b390329e-625b-4236-a493-adb9ed8061cd\") "
Jul 30 20:27:41.387023 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.386939    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9hfr\" (UniqueName: \"kubernetes.io/projected/f3ffd11c-0d7b-4529-8edf-0925972813fd-kube-api-access-k9hfr\") pod \"f3ffd11c-0d7b-4529-8edf-0925972813fd\" (UID: \"f3ffd11c-0d7b-4529-8edf-0925972813fd\") "
Jul 30 20:27:41.387023 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.387003    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b390329e-625b-4236-a493-adb9ed8061cd-openshift-service-ca\") pod \"b390329e-625b-4236-a493-adb9ed8061cd\" (UID: \"b390329e-625b-4236-a493-adb9ed8061cd\") "
Jul 30 20:27:41.387314 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.387288    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3ffd11c-0d7b-4529-8edf-0925972813fd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f3ffd11c-0d7b-4529-8edf-0925972813fd" (UID: "f3ffd11c-0d7b-4529-8edf-0925972813fd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:27:41.387668 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.387627    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b390329e-625b-4236-a493-adb9ed8061cd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b390329e-625b-4236-a493-adb9ed8061cd" (UID: "b390329e-625b-4236-a493-adb9ed8061cd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:27:41.389213 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.389179    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3ffd11c-0d7b-4529-8edf-0925972813fd-kube-api-access-k9hfr" (OuterVolumeSpecName: "kube-api-access-k9hfr") pod "f3ffd11c-0d7b-4529-8edf-0925972813fd" (UID: "f3ffd11c-0d7b-4529-8edf-0925972813fd"). InnerVolumeSpecName "kube-api-access-k9hfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:27:41.389637 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.389601    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b390329e-625b-4236-a493-adb9ed8061cd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b390329e-625b-4236-a493-adb9ed8061cd" (UID: "b390329e-625b-4236-a493-adb9ed8061cd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:27:41.389735 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.389637    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b390329e-625b-4236-a493-adb9ed8061cd-kube-api-access-998jp" (OuterVolumeSpecName: "kube-api-access-998jp") pod "b390329e-625b-4236-a493-adb9ed8061cd" (UID: "b390329e-625b-4236-a493-adb9ed8061cd"). InnerVolumeSpecName "kube-api-access-998jp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:27:41.488564 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.488539    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-998jp\" (UniqueName: \"kubernetes.io/projected/b390329e-625b-4236-a493-adb9ed8061cd-kube-api-access-998jp\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:27:41.488564 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.488562    2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3ffd11c-0d7b-4529-8edf-0925972813fd-plugins-config-volume\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:27:41.488743 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.488571    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b390329e-625b-4236-a493-adb9ed8061cd-maas-api-tls\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:27:41.488743 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.488579    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9hfr\" (UniqueName: \"kubernetes.io/projected/f3ffd11c-0d7b-4529-8edf-0925972813fd-kube-api-access-k9hfr\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:27:41.488743 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.488591    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b390329e-625b-4236-a493-adb9ed8061cd-openshift-service-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:27:41.527755 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.527720    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:27:41.713849 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.713811    2554 generic.go:356] "Generic (PLEG): container finished" podID="b390329e-625b-4236-a493-adb9ed8061cd" containerID="36e229fd5fc8bcc014ce0940d605a91a1543d6da19143970d910ebde6cb52c66" exitCode=0
Jul 30 20:27:41.714058 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.713854    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v"
Jul 30 20:27:41.714058 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.713887    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v" event={"ID":"b390329e-625b-4236-a493-adb9ed8061cd","Type":"ContainerDied","Data":"36e229fd5fc8bcc014ce0940d605a91a1543d6da19143970d910ebde6cb52c66"}
Jul 30 20:27:41.714058 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.713977    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v" event={"ID":"b390329e-625b-4236-a493-adb9ed8061cd","Type":"ContainerDied","Data":"80b0efc69ed452aaf4204274bb0c7153ae8d4981340b930661c5d8263549938f"}
Jul 30 20:27:41.714058 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.714009    2554 scope.go:117] "RemoveContainer" containerID="36e229fd5fc8bcc014ce0940d605a91a1543d6da19143970d910ebde6cb52c66"
Jul 30 20:27:41.715579 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.715555    2554 generic.go:356] "Generic (PLEG): container finished" podID="f3ffd11c-0d7b-4529-8edf-0925972813fd" containerID="8f0a9138744aff2aeca72dc05474e7b752ff1254498f3c24551461d41db99ff1" exitCode=0
Jul 30 20:27:41.715745 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.715599    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc"
Jul 30 20:27:41.715745 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.715684    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc" event={"ID":"f3ffd11c-0d7b-4529-8edf-0925972813fd","Type":"ContainerDied","Data":"8f0a9138744aff2aeca72dc05474e7b752ff1254498f3c24551461d41db99ff1"}
Jul 30 20:27:41.715745 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.715728    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc" event={"ID":"f3ffd11c-0d7b-4529-8edf-0925972813fd","Type":"ContainerDied","Data":"8619b5c96e592168a9ab74bb7b002f60ea7c23c347e233c095b3a94c3f0691cf"}
Jul 30 20:27:41.725298 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.725276    2554 scope.go:117] "RemoveContainer" containerID="36e229fd5fc8bcc014ce0940d605a91a1543d6da19143970d910ebde6cb52c66"
Jul 30 20:27:41.725604 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:27:41.725587    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36e229fd5fc8bcc014ce0940d605a91a1543d6da19143970d910ebde6cb52c66\": container with ID starting with 36e229fd5fc8bcc014ce0940d605a91a1543d6da19143970d910ebde6cb52c66 not found: ID does not exist" containerID="36e229fd5fc8bcc014ce0940d605a91a1543d6da19143970d910ebde6cb52c66"
Jul 30 20:27:41.725654 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.725614    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36e229fd5fc8bcc014ce0940d605a91a1543d6da19143970d910ebde6cb52c66"} err="failed to get container status \"36e229fd5fc8bcc014ce0940d605a91a1543d6da19143970d910ebde6cb52c66\": rpc error: code = NotFound desc = could not find container \"36e229fd5fc8bcc014ce0940d605a91a1543d6da19143970d910ebde6cb52c66\": container with ID starting with 36e229fd5fc8bcc014ce0940d605a91a1543d6da19143970d910ebde6cb52c66 not found: ID does not exist"
Jul 30 20:27:41.725654 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.725630    2554 scope.go:117] "RemoveContainer" containerID="8f0a9138744aff2aeca72dc05474e7b752ff1254498f3c24551461d41db99ff1"
Jul 30 20:27:41.734478 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.734458    2554 scope.go:117] "RemoveContainer" containerID="8f0a9138744aff2aeca72dc05474e7b752ff1254498f3c24551461d41db99ff1"
Jul 30 20:27:41.734989 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:27:41.734946    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f0a9138744aff2aeca72dc05474e7b752ff1254498f3c24551461d41db99ff1\": container with ID starting with 8f0a9138744aff2aeca72dc05474e7b752ff1254498f3c24551461d41db99ff1 not found: ID does not exist" containerID="8f0a9138744aff2aeca72dc05474e7b752ff1254498f3c24551461d41db99ff1"
Jul 30 20:27:41.735094 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.734998    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f0a9138744aff2aeca72dc05474e7b752ff1254498f3c24551461d41db99ff1"} err="failed to get container status \"8f0a9138744aff2aeca72dc05474e7b752ff1254498f3c24551461d41db99ff1\": rpc error: code = NotFound desc = could not find container \"8f0a9138744aff2aeca72dc05474e7b752ff1254498f3c24551461d41db99ff1\": container with ID starting with 8f0a9138744aff2aeca72dc05474e7b752ff1254498f3c24551461d41db99ff1 not found: ID does not exist"
Jul 30 20:27:41.737661 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.737641    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v"]
Jul 30 20:27:41.741196 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.741138    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d67d52-854f8d4875-xcm8v"]
Jul 30 20:27:41.750625 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.750604    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc"]
Jul 30 20:27:41.753987 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:41.753964    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d67d52-5b8dcd96-vm7mc"]
Jul 30 20:27:42.486528 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:42.486492    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80fed8d0-d0d6-47b5-8765-cc6dad641f14" path="/var/lib/kubelet/pods/80fed8d0-d0d6-47b5-8765-cc6dad641f14/volumes"
Jul 30 20:27:42.487118 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:42.487100    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b390329e-625b-4236-a493-adb9ed8061cd" path="/var/lib/kubelet/pods/b390329e-625b-4236-a493-adb9ed8061cd/volumes"
Jul 30 20:27:42.487504 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:42.487485    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3ffd11c-0d7b-4529-8edf-0925972813fd" path="/var/lib/kubelet/pods/f3ffd11c-0d7b-4529-8edf-0925972813fd/volumes"
Jul 30 20:27:47.189996 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:47.189929    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:27:47.247611 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:47.247581    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:27:47.481921 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:47.481886    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:27:47.699436 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:47.699404    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:27:47.841997 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:47.841895    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ecda00-x5rwz"]
Jul 30 20:27:47.842360 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:47.842344    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3ffd11c-0d7b-4529-8edf-0925972813fd" containerName="payload-processing"
Jul 30 20:27:47.842415 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:47.842362    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3ffd11c-0d7b-4529-8edf-0925972813fd" containerName="payload-processing"
Jul 30 20:27:47.842415 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:47.842370    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80fed8d0-d0d6-47b5-8765-cc6dad641f14" containerName="revoke-keys"
Jul 30 20:27:47.842415 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:47.842376    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="80fed8d0-d0d6-47b5-8765-cc6dad641f14" containerName="revoke-keys"
Jul 30 20:27:47.842415 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:47.842397    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b390329e-625b-4236-a493-adb9ed8061cd" containerName="maas-api"
Jul 30 20:27:47.842415 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:47.842404    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="b390329e-625b-4236-a493-adb9ed8061cd" containerName="maas-api"
Jul 30 20:27:47.842597 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:47.842478    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="80fed8d0-d0d6-47b5-8765-cc6dad641f14" containerName="revoke-keys"
Jul 30 20:27:47.842597 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:47.842486    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="b390329e-625b-4236-a493-adb9ed8061cd" containerName="maas-api"
Jul 30 20:27:47.842597 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:47.842492    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3ffd11c-0d7b-4529-8edf-0925972813fd" containerName="payload-processing"
Jul 30 20:27:47.847201 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:47.847184    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ecda00-x5rwz"
Jul 30 20:27:47.849722 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:47.849698    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7t7kk\""
Jul 30 20:27:47.854266 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:47.853672    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ecda00-x5rwz"]
Jul 30 20:27:47.955478 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:47.955437    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d326f6e6-cf17-462b-ae41-0e0d346bdeb4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-ecda00-x5rwz\" (UID: \"d326f6e6-cf17-462b-ae41-0e0d346bdeb4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ecda00-x5rwz"
Jul 30 20:27:48.057079 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:48.057043    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d326f6e6-cf17-462b-ae41-0e0d346bdeb4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-ecda00-x5rwz\" (UID: \"d326f6e6-cf17-462b-ae41-0e0d346bdeb4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ecda00-x5rwz"
Jul 30 20:27:48.057654 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:48.057635    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d326f6e6-cf17-462b-ae41-0e0d346bdeb4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-ecda00-x5rwz\" (UID: \"d326f6e6-cf17-462b-ae41-0e0d346bdeb4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ecda00-x5rwz"
Jul 30 20:27:48.162250 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:48.162165    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ecda00-x5rwz"
Jul 30 20:27:48.303693 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:48.303663    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ecda00-x5rwz"]
Jul 30 20:27:48.305208 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:27:48.305180    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd326f6e6_cf17_462b_ae41_0e0d346bdeb4.slice/crio-deeb171485d31c34c000e4b2882a12464ad11f057b035c66b44aff7b3357086b WatchSource:0}: Error finding container deeb171485d31c34c000e4b2882a12464ad11f057b035c66b44aff7b3357086b: Status 404 returned error can't find the container with id deeb171485d31c34c000e4b2882a12464ad11f057b035c66b44aff7b3357086b
Jul 30 20:27:48.764505 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:48.764465    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ecda00-x5rwz" event={"ID":"d326f6e6-cf17-462b-ae41-0e0d346bdeb4","Type":"ContainerStarted","Data":"dc531e4fca5a0e6735b2e98631ecc2c4f773e621752f5a6a8ec75387d100e503"}
Jul 30 20:27:48.764718 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:48.764525    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ecda00-x5rwz" event={"ID":"d326f6e6-cf17-462b-ae41-0e0d346bdeb4","Type":"ContainerStarted","Data":"deeb171485d31c34c000e4b2882a12464ad11f057b035c66b44aff7b3357086b"}
Jul 30 20:27:48.780282 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:48.780184    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ecda00-x5rwz" podStartSLOduration=1.780169813 podStartE2EDuration="1.780169813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 20:27:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:27:48.778400566 +0000 UTC m=+3796.742981648" watchObservedRunningTime="2026-07-30 20:27:48.780169813 +0000 UTC m=+3796.744750893"
Jul 30 20:27:49.772014 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:49.771974    2554 generic.go:356] "Generic (PLEG): container finished" podID="d326f6e6-cf17-462b-ae41-0e0d346bdeb4" containerID="dc531e4fca5a0e6735b2e98631ecc2c4f773e621752f5a6a8ec75387d100e503" exitCode=0
Jul 30 20:27:49.772412 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:49.772044    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ecda00-x5rwz" event={"ID":"d326f6e6-cf17-462b-ae41-0e0d346bdeb4","Type":"ContainerDied","Data":"dc531e4fca5a0e6735b2e98631ecc2c4f773e621752f5a6a8ec75387d100e503"}
Jul 30 20:27:50.916158 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:50.916132    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ecda00-x5rwz"
Jul 30 20:27:51.089120 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:51.089031    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d326f6e6-cf17-462b-ae41-0e0d346bdeb4-maas-api-service-ca\") pod \"d326f6e6-cf17-462b-ae41-0e0d346bdeb4\" (UID: \"d326f6e6-cf17-462b-ae41-0e0d346bdeb4\") "
Jul 30 20:27:51.089447 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:51.089417    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d326f6e6-cf17-462b-ae41-0e0d346bdeb4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d326f6e6-cf17-462b-ae41-0e0d346bdeb4" (UID: "d326f6e6-cf17-462b-ae41-0e0d346bdeb4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:27:51.190209 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:51.190169    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d326f6e6-cf17-462b-ae41-0e0d346bdeb4-maas-api-service-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:27:51.785545 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:51.785510    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ecda00-x5rwz" event={"ID":"d326f6e6-cf17-462b-ae41-0e0d346bdeb4","Type":"ContainerDied","Data":"deeb171485d31c34c000e4b2882a12464ad11f057b035c66b44aff7b3357086b"}
Jul 30 20:27:51.785724 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:51.785562    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="deeb171485d31c34c000e4b2882a12464ad11f057b035c66b44aff7b3357086b"
Jul 30 20:27:51.785724 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:51.785519    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ecda00-x5rwz"
Jul 30 20:27:58.224492 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.224454    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n"]
Jul 30 20:27:58.225013 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.224709    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n" podUID="811a3018-aa91-4f66-9dfd-2eb7baf8c3d5" containerName="maas-api" containerID="cri-o://1f8aa6787ebfd22126d9ef50c6041891362840c5f5f3aa0d719d1b8ce1e031c0" gracePeriod=30
Jul 30 20:27:58.249227 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.249198    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:27:58.255678 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.255652    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ecda00-29757375-ln47x"]
Jul 30 20:27:58.258274 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.258253    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ecda00-29757375-ln47x"]
Jul 30 20:27:58.321226 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.321184    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:27:58.373177 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.373129    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:27:58.416179 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.416113    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ecda00-x5rwz"]
Jul 30 20:27:58.419391 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.419362    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ecda00-x5rwz"]
Jul 30 20:27:58.431061 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.431034    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:27:58.483019 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.482920    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n"
Jul 30 20:27:58.487025 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.487001    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2061f5b-ff8d-4473-9f9f-eaac8f123f5b" path="/var/lib/kubelet/pods/c2061f5b-ff8d-4473-9f9f-eaac8f123f5b/volumes"
Jul 30 20:27:58.487366 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.487353    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d326f6e6-cf17-462b-ae41-0e0d346bdeb4" path="/var/lib/kubelet/pods/d326f6e6-cf17-462b-ae41-0e0d346bdeb4/volumes"
Jul 30 20:27:58.558905 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.558870    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/811a3018-aa91-4f66-9dfd-2eb7baf8c3d5-maas-api-tls\") pod \"811a3018-aa91-4f66-9dfd-2eb7baf8c3d5\" (UID: \"811a3018-aa91-4f66-9dfd-2eb7baf8c3d5\") "
Jul 30 20:27:58.559118 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.558978    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/811a3018-aa91-4f66-9dfd-2eb7baf8c3d5-openshift-service-ca\") pod \"811a3018-aa91-4f66-9dfd-2eb7baf8c3d5\" (UID: \"811a3018-aa91-4f66-9dfd-2eb7baf8c3d5\") "
Jul 30 20:27:58.559118 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.559031    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ssgwr\" (UniqueName: \"kubernetes.io/projected/811a3018-aa91-4f66-9dfd-2eb7baf8c3d5-kube-api-access-ssgwr\") pod \"811a3018-aa91-4f66-9dfd-2eb7baf8c3d5\" (UID: \"811a3018-aa91-4f66-9dfd-2eb7baf8c3d5\") "
Jul 30 20:27:58.559400 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.559372    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/811a3018-aa91-4f66-9dfd-2eb7baf8c3d5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "811a3018-aa91-4f66-9dfd-2eb7baf8c3d5" (UID: "811a3018-aa91-4f66-9dfd-2eb7baf8c3d5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:27:58.561152 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.561130    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/811a3018-aa91-4f66-9dfd-2eb7baf8c3d5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "811a3018-aa91-4f66-9dfd-2eb7baf8c3d5" (UID: "811a3018-aa91-4f66-9dfd-2eb7baf8c3d5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 20:27:58.561231 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.561161    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/811a3018-aa91-4f66-9dfd-2eb7baf8c3d5-kube-api-access-ssgwr" (OuterVolumeSpecName: "kube-api-access-ssgwr") pod "811a3018-aa91-4f66-9dfd-2eb7baf8c3d5" (UID: "811a3018-aa91-4f66-9dfd-2eb7baf8c3d5"). InnerVolumeSpecName "kube-api-access-ssgwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:27:58.660437 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.660403    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/811a3018-aa91-4f66-9dfd-2eb7baf8c3d5-openshift-service-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:27:58.660437 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.660433    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssgwr\" (UniqueName: \"kubernetes.io/projected/811a3018-aa91-4f66-9dfd-2eb7baf8c3d5-kube-api-access-ssgwr\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:27:58.660437 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.660443    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/811a3018-aa91-4f66-9dfd-2eb7baf8c3d5-maas-api-tls\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:27:58.794196 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.794109    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:27:58.838063 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.838024    2554 generic.go:356] "Generic (PLEG): container finished" podID="811a3018-aa91-4f66-9dfd-2eb7baf8c3d5" containerID="1f8aa6787ebfd22126d9ef50c6041891362840c5f5f3aa0d719d1b8ce1e031c0" exitCode=0
Jul 30 20:27:58.838063 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.838067    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n"
Jul 30 20:27:58.838270 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.838117    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n" event={"ID":"811a3018-aa91-4f66-9dfd-2eb7baf8c3d5","Type":"ContainerDied","Data":"1f8aa6787ebfd22126d9ef50c6041891362840c5f5f3aa0d719d1b8ce1e031c0"}
Jul 30 20:27:58.838270 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.838168    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n" event={"ID":"811a3018-aa91-4f66-9dfd-2eb7baf8c3d5","Type":"ContainerDied","Data":"216e4f0683e6b558a7162798093d7bb9f41d46cd8ca551a45fa0f9d05c1d1d1d"}
Jul 30 20:27:58.838270 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.838195    2554 scope.go:117] "RemoveContainer" containerID="1f8aa6787ebfd22126d9ef50c6041891362840c5f5f3aa0d719d1b8ce1e031c0"
Jul 30 20:27:58.848118 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.848101    2554 scope.go:117] "RemoveContainer" containerID="1f8aa6787ebfd22126d9ef50c6041891362840c5f5f3aa0d719d1b8ce1e031c0"
Jul 30 20:27:58.848391 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:27:58.848362    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f8aa6787ebfd22126d9ef50c6041891362840c5f5f3aa0d719d1b8ce1e031c0\": container with ID starting with 1f8aa6787ebfd22126d9ef50c6041891362840c5f5f3aa0d719d1b8ce1e031c0 not found: ID does not exist" containerID="1f8aa6787ebfd22126d9ef50c6041891362840c5f5f3aa0d719d1b8ce1e031c0"
Jul 30 20:27:58.848467 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.848402    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f8aa6787ebfd22126d9ef50c6041891362840c5f5f3aa0d719d1b8ce1e031c0"} err="failed to get container status \"1f8aa6787ebfd22126d9ef50c6041891362840c5f5f3aa0d719d1b8ce1e031c0\": rpc error: code = NotFound desc = could not find container \"1f8aa6787ebfd22126d9ef50c6041891362840c5f5f3aa0d719d1b8ce1e031c0\": container with ID starting with 1f8aa6787ebfd22126d9ef50c6041891362840c5f5f3aa0d719d1b8ce1e031c0 not found: ID does not exist"
Jul 30 20:27:58.859497 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.859471    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n"]
Jul 30 20:27:58.864870 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:27:58.864849    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ecda00-779bf88b87-rdj6n"]
Jul 30 20:28:00.486685 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:00.486653    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="811a3018-aa91-4f66-9dfd-2eb7baf8c3d5" path="/var/lib/kubelet/pods/811a3018-aa91-4f66-9dfd-2eb7baf8c3d5/volumes"
Jul 30 20:28:04.724349 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:04.724317    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:28:04.820456 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:04.820422    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:28:04.970385 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:04.970351    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"]
Jul 30 20:28:04.970586 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:04.970560    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc" podUID="2c9ed2ee-aae2-41dc-b31b-3207dab9e52e" containerName="istio-proxy" containerID="cri-o://046f66345afe96ef9f0c7e35e2480e9a51ab02732bb85757a764d8043aafa00b" gracePeriod=30
Jul 30 20:28:05.007021 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:05.006885    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:28:05.229887 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:05.229856    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:28:05.343761 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:05.343667    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a769e5-xkzs5"]
Jul 30 20:28:05.344129 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:05.344115    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d326f6e6-cf17-462b-ae41-0e0d346bdeb4" containerName="revoke-keys"
Jul 30 20:28:05.344181 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:05.344131    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="d326f6e6-cf17-462b-ae41-0e0d346bdeb4" containerName="revoke-keys"
Jul 30 20:28:05.344181 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:05.344149    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="811a3018-aa91-4f66-9dfd-2eb7baf8c3d5" containerName="maas-api"
Jul 30 20:28:05.344181 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:05.344155    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="811a3018-aa91-4f66-9dfd-2eb7baf8c3d5" containerName="maas-api"
Jul 30 20:28:05.344279 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:05.344219    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="811a3018-aa91-4f66-9dfd-2eb7baf8c3d5" containerName="maas-api"
Jul 30 20:28:05.344279 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:05.344230    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="d326f6e6-cf17-462b-ae41-0e0d346bdeb4" containerName="revoke-keys"
Jul 30 20:28:05.348673 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:05.348656    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a769e5-xkzs5"
Jul 30 20:28:05.351919 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:05.351897    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7t7kk\""
Jul 30 20:28:05.358919 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:05.358898    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a769e5-xkzs5"]
Jul 30 20:28:05.424854 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:05.424812    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12138b79-34b5-4bec-96dd-4204d00ee378-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-a769e5-xkzs5\" (UID: \"12138b79-34b5-4bec-96dd-4204d00ee378\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a769e5-xkzs5"
Jul 30 20:28:05.526485 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:05.526452    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12138b79-34b5-4bec-96dd-4204d00ee378-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-a769e5-xkzs5\" (UID: \"12138b79-34b5-4bec-96dd-4204d00ee378\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a769e5-xkzs5"
Jul 30 20:28:05.527185 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:05.527160    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12138b79-34b5-4bec-96dd-4204d00ee378-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-a769e5-xkzs5\" (UID: \"12138b79-34b5-4bec-96dd-4204d00ee378\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a769e5-xkzs5"
Jul 30 20:28:05.661297 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:05.661214    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a769e5-xkzs5"
Jul 30 20:28:05.804639 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:05.804613    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a769e5-xkzs5"]
Jul 30 20:28:05.807356 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:28:05.807303    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12138b79_34b5_4bec_96dd_4204d00ee378.slice/crio-d34991a726f15d328ca6351119b07eefb59ba31692bb5578bca0c413c4e2ef1a WatchSource:0}: Error finding container d34991a726f15d328ca6351119b07eefb59ba31692bb5578bca0c413c4e2ef1a: Status 404 returned error can't find the container with id d34991a726f15d328ca6351119b07eefb59ba31692bb5578bca0c413c4e2ef1a
Jul 30 20:28:05.883448 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:05.883410    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a769e5-xkzs5" event={"ID":"12138b79-34b5-4bec-96dd-4204d00ee378","Type":"ContainerStarted","Data":"d34991a726f15d328ca6351119b07eefb59ba31692bb5578bca0c413c4e2ef1a"}
Jul 30 20:28:06.891901 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:06.891813    2554 generic.go:356] "Generic (PLEG): container finished" podID="12138b79-34b5-4bec-96dd-4204d00ee378" containerID="827e543a8edaf00256dd9f5be56e4d2a5f83b42be05337076d431639bda49af5" exitCode=0
Jul 30 20:28:06.891901 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:06.891862    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a769e5-xkzs5" event={"ID":"12138b79-34b5-4bec-96dd-4204d00ee378","Type":"ContainerDied","Data":"827e543a8edaf00256dd9f5be56e4d2a5f83b42be05337076d431639bda49af5"}
Jul 30 20:28:08.032922 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:08.032899    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a769e5-xkzs5"
Jul 30 20:28:08.152339 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:08.152239    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12138b79-34b5-4bec-96dd-4204d00ee378-maas-api-service-ca\") pod \"12138b79-34b5-4bec-96dd-4204d00ee378\" (UID: \"12138b79-34b5-4bec-96dd-4204d00ee378\") "
Jul 30 20:28:08.152691 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:08.152657    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12138b79-34b5-4bec-96dd-4204d00ee378-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "12138b79-34b5-4bec-96dd-4204d00ee378" (UID: "12138b79-34b5-4bec-96dd-4204d00ee378"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:28:08.253288 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:08.253253    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12138b79-34b5-4bec-96dd-4204d00ee378-maas-api-service-ca\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:28:08.906311 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:08.906224    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a769e5-xkzs5"
Jul 30 20:28:08.906311 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:08.906261    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a769e5-xkzs5" event={"ID":"12138b79-34b5-4bec-96dd-4204d00ee378","Type":"ContainerDied","Data":"d34991a726f15d328ca6351119b07eefb59ba31692bb5578bca0c413c4e2ef1a"}
Jul 30 20:28:08.906311 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:08.906307    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d34991a726f15d328ca6351119b07eefb59ba31692bb5578bca0c413c4e2ef1a"
Jul 30 20:28:10.238810 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.238749    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:28:10.373739 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.373703    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-credential-socket\") pod \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") "
Jul 30 20:28:10.373739 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.373743    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-workload-socket\") pod \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") "
Jul 30 20:28:10.374008 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.373760    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istio-podinfo\") pod \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") "
Jul 30 20:28:10.374008 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.373798    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istiod-ca-cert\") pod \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") "
Jul 30 20:28:10.374008 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.373832    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istio-envoy\") pod \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") "
Jul 30 20:28:10.374008 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.373882    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istio-data\") pod \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") "
Jul 30 20:28:10.374008 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.373938    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2k2qw\" (UniqueName: \"kubernetes.io/projected/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-kube-api-access-2k2qw\") pod \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") "
Jul 30 20:28:10.374008 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.373995    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istio-token\") pod \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") "
Jul 30 20:28:10.374382 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.374035    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2c9ed2ee-aae2-41dc-b31b-3207dab9e52e" (UID: "2c9ed2ee-aae2-41dc-b31b-3207dab9e52e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:28:10.374382 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.374051    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-workload-certs\") pod \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\" (UID: \"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e\") "
Jul 30 20:28:10.374382 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.374090    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2c9ed2ee-aae2-41dc-b31b-3207dab9e52e" (UID: "2c9ed2ee-aae2-41dc-b31b-3207dab9e52e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:28:10.374382 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.374271    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2c9ed2ee-aae2-41dc-b31b-3207dab9e52e" (UID: "2c9ed2ee-aae2-41dc-b31b-3207dab9e52e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:28:10.374382 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.374292    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2c9ed2ee-aae2-41dc-b31b-3207dab9e52e" (UID: "2c9ed2ee-aae2-41dc-b31b-3207dab9e52e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 20:28:10.374661 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.374508    2554 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-workload-certs\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:28:10.374661 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.374529    2554 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-credential-socket\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:28:10.374661 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.374544    2554 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-workload-socket\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:28:10.374661 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.374559    2554 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istiod-ca-cert\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:28:10.375082 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.375058    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istio-data" (OuterVolumeSpecName: "istio-data") pod "2c9ed2ee-aae2-41dc-b31b-3207dab9e52e" (UID: "2c9ed2ee-aae2-41dc-b31b-3207dab9e52e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:28:10.376101 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.376076    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2c9ed2ee-aae2-41dc-b31b-3207dab9e52e" (UID: "2c9ed2ee-aae2-41dc-b31b-3207dab9e52e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 20:28:10.376206 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.376192    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-kube-api-access-2k2qw" (OuterVolumeSpecName: "kube-api-access-2k2qw") pod "2c9ed2ee-aae2-41dc-b31b-3207dab9e52e" (UID: "2c9ed2ee-aae2-41dc-b31b-3207dab9e52e"). InnerVolumeSpecName "kube-api-access-2k2qw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:28:10.376462 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.376445    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istio-token" (OuterVolumeSpecName: "istio-token") pod "2c9ed2ee-aae2-41dc-b31b-3207dab9e52e" (UID: "2c9ed2ee-aae2-41dc-b31b-3207dab9e52e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 20:28:10.376462 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.376446    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2c9ed2ee-aae2-41dc-b31b-3207dab9e52e" (UID: "2c9ed2ee-aae2-41dc-b31b-3207dab9e52e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 20:28:10.475674 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.475639    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2k2qw\" (UniqueName: \"kubernetes.io/projected/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-kube-api-access-2k2qw\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:28:10.475674 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.475669    2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istio-token\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:28:10.475866 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.475684    2554 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istio-podinfo\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:28:10.475866 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.475696    2554 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istio-envoy\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:28:10.475866 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.475707    2554 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e-istio-data\") on node \"ip-10-0-134-180.ec2.internal\" DevicePath \"\""
Jul 30 20:28:10.921989 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.921890    2554 generic.go:356] "Generic (PLEG): container finished" podID="2c9ed2ee-aae2-41dc-b31b-3207dab9e52e" containerID="046f66345afe96ef9f0c7e35e2480e9a51ab02732bb85757a764d8043aafa00b" exitCode=0
Jul 30 20:28:10.921989 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.921936    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"
Jul 30 20:28:10.922170 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.921982    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc" event={"ID":"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e","Type":"ContainerDied","Data":"046f66345afe96ef9f0c7e35e2480e9a51ab02732bb85757a764d8043aafa00b"}
Jul 30 20:28:10.922170 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.922040    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc" event={"ID":"2c9ed2ee-aae2-41dc-b31b-3207dab9e52e","Type":"ContainerDied","Data":"3e2e515d69cb0f2a164f9d674af92a5958bdf1b4f535bd30fee8c8b8c6d7c8ce"}
Jul 30 20:28:10.922170 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.922068    2554 scope.go:117] "RemoveContainer" containerID="046f66345afe96ef9f0c7e35e2480e9a51ab02732bb85757a764d8043aafa00b"
Jul 30 20:28:10.931576 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.931559    2554 scope.go:117] "RemoveContainer" containerID="f569cf59185260ac3ad0023d2c4fa514b7fce9f2f3ca7d969a545eb82681ad57"
Jul 30 20:28:10.940370 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.940347    2554 scope.go:117] "RemoveContainer" containerID="046f66345afe96ef9f0c7e35e2480e9a51ab02732bb85757a764d8043aafa00b"
Jul 30 20:28:10.940534 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.940508    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"]
Jul 30 20:28:10.940643 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:28:10.940621    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"046f66345afe96ef9f0c7e35e2480e9a51ab02732bb85757a764d8043aafa00b\": container with ID starting with 046f66345afe96ef9f0c7e35e2480e9a51ab02732bb85757a764d8043aafa00b not found: ID does not exist" containerID="046f66345afe96ef9f0c7e35e2480e9a51ab02732bb85757a764d8043aafa00b"
Jul 30 20:28:10.940693 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.940652    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"046f66345afe96ef9f0c7e35e2480e9a51ab02732bb85757a764d8043aafa00b"} err="failed to get container status \"046f66345afe96ef9f0c7e35e2480e9a51ab02732bb85757a764d8043aafa00b\": rpc error: code = NotFound desc = could not find container \"046f66345afe96ef9f0c7e35e2480e9a51ab02732bb85757a764d8043aafa00b\": container with ID starting with 046f66345afe96ef9f0c7e35e2480e9a51ab02732bb85757a764d8043aafa00b not found: ID does not exist"
Jul 30 20:28:10.940693 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.940669    2554 scope.go:117] "RemoveContainer" containerID="f569cf59185260ac3ad0023d2c4fa514b7fce9f2f3ca7d969a545eb82681ad57"
Jul 30 20:28:10.940926 ip-10-0-134-180 kubenswrapper[2554]: E0730 20:28:10.940909    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f569cf59185260ac3ad0023d2c4fa514b7fce9f2f3ca7d969a545eb82681ad57\": container with ID starting with f569cf59185260ac3ad0023d2c4fa514b7fce9f2f3ca7d969a545eb82681ad57 not found: ID does not exist" containerID="f569cf59185260ac3ad0023d2c4fa514b7fce9f2f3ca7d969a545eb82681ad57"
Jul 30 20:28:10.941010 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.940939    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f569cf59185260ac3ad0023d2c4fa514b7fce9f2f3ca7d969a545eb82681ad57"} err="failed to get container status \"f569cf59185260ac3ad0023d2c4fa514b7fce9f2f3ca7d969a545eb82681ad57\": rpc error: code = NotFound desc = could not find container \"f569cf59185260ac3ad0023d2c4fa514b7fce9f2f3ca7d969a545eb82681ad57\": container with ID starting with f569cf59185260ac3ad0023d2c4fa514b7fce9f2f3ca7d969a545eb82681ad57 not found: ID does not exist"
Jul 30 20:28:10.945791 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:10.945771    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-ecda00-openshift-default-fdc977b6d-x42nc"]
Jul 30 20:28:12.487043 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:12.487001    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c9ed2ee-aae2-41dc-b31b-3207dab9e52e" path="/var/lib/kubelet/pods/2c9ed2ee-aae2-41dc-b31b-3207dab9e52e/volumes"
Jul 30 20:28:15.759709 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:15.759678    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:28:15.765272 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:15.765239    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a769e5-29757375-g74gj"]
Jul 30 20:28:15.768132 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:15.768109    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a769e5-29757375-g74gj"]
Jul 30 20:28:15.825721 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:15.825682    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:28:15.925891 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:15.925861    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:28:15.938381 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:15.938355    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a769e5-xkzs5"]
Jul 30 20:28:15.940724 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:15.940702    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a769e5-xkzs5"]
Jul 30 20:28:15.994206 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:15.994171    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:28:16.335668 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:16.335636    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:28:16.486885 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:16.486842    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12138b79-34b5-4bec-96dd-4204d00ee378" path="/var/lib/kubelet/pods/12138b79-34b5-4bec-96dd-4204d00ee378/volumes"
Jul 30 20:28:16.487288 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:16.487267    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdbcb8ce-e98a-4894-9fa4-3febd0fbef54" path="/var/lib/kubelet/pods/bdbcb8ce-e98a-4894-9fa4-3febd0fbef54/volumes"
Jul 30 20:28:22.383421 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:22.383384    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:28:23.061627 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:23.061592    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:28:23.132779 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:23.132744    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:28:28.874926 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:28.874897    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-785db54bd5-c59kr_f9e40da3-e49e-4085-8d78-749bc2f9394c/manager/0.log"
Jul 30 20:28:29.355166 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:29.355138    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr_a1391f7f-8cee-46c8-940f-167d78cd70e3/copy-manifests/0.log"
Jul 30 20:28:29.362274 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:29.362256    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr_a1391f7f-8cee-46c8-940f-167d78cd70e3/manager/0.log"
Jul 30 20:28:29.829519 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:29.829489    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-844cc9dc96-gs8n7_dafdfcb4-7499-42ec-b9ed-5856bc2f5eaa/maas-api/0.log"
Jul 30 20:28:29.941827 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:29.941802    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29757345-v7wzk_89254ead-4e44-461c-a98e-31d90cf9d972/cleanup/0.log"
Jul 30 20:28:30.055319 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:30.055293    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29757360-kptpq_dc61d13c-d148-4670-a971-17df143d3509/cleanup/0.log"
Jul 30 20:28:30.169589 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:30.169510    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29757375-bfgnl_446a5558-e343-48f5-bb07-a0a8e1dd9cf2/cleanup/0.log"
Jul 30 20:28:30.284208 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:30.284176    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-8977m_ca6f2c34-27da-488d-a73e-27f0cf705c92/postgres/0.log"
Jul 30 20:28:31.069413 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:31.069388    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx_2a84b32c-a92e-45e6-ae5c-6a59c1618028/pull/0.log"
Jul 30 20:28:31.076302 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:31.076282    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx_2a84b32c-a92e-45e6-ae5c-6a59c1618028/extract/0.log"
Jul 30 20:28:31.082987 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:31.082968    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx_2a84b32c-a92e-45e6-ae5c-6a59c1618028/util/0.log"
Jul 30 20:28:31.196411 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:31.196387    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr_32b10063-97db-47e5-bd85-d3dd6633e9f5/util/0.log"
Jul 30 20:28:31.203272 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:31.203250    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr_32b10063-97db-47e5-bd85-d3dd6633e9f5/pull/0.log"
Jul 30 20:28:31.210297 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:31.210276    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr_32b10063-97db-47e5-bd85-d3dd6633e9f5/extract/0.log"
Jul 30 20:28:31.319789 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:31.319723    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4_e0266cc8-8703-4da9-9029-2170fdf22e9d/util/0.log"
Jul 30 20:28:31.326220 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:31.326193    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4_e0266cc8-8703-4da9-9029-2170fdf22e9d/pull/0.log"
Jul 30 20:28:31.332534 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:31.332510    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4_e0266cc8-8703-4da9-9029-2170fdf22e9d/extract/0.log"
Jul 30 20:28:31.449084 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:31.449055    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq_29bcabf6-73c0-47a5-8d47-0a92f7a945f7/pull/0.log"
Jul 30 20:28:31.455837 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:31.455818    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq_29bcabf6-73c0-47a5-8d47-0a92f7a945f7/extract/0.log"
Jul 30 20:28:31.462183 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:31.462165    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq_29bcabf6-73c0-47a5-8d47-0a92f7a945f7/util/0.log"
Jul 30 20:28:31.811253 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:31.811227    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-d9nzc_ace7990c-649f-496b-8cd1-52ca9a626f9f/manager/0.log"
Jul 30 20:28:31.919376 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:31.919350    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-sv8wn_502f8628-f1d0-4b2d-908a-8b09e2211e59/kuadrant-console-plugin/0.log"
Jul 30 20:28:32.036882 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:32.036846    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-rmxcj_c2145a09-d30d-4832-9b2d-40f1beebb16c/registry-server/0.log"
Jul 30 20:28:32.158151 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:32.158073    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-w5ffc_e7accbac-7008-46b2-b1c1-37a50151e649/manager/0.log"
Jul 30 20:28:32.742325 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:32.742297    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv_2c97707f-d73f-4d27-84f7-3152b9b709ea/istio-proxy/0.log"
Jul 30 20:28:33.204715 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:33.204639    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-fbm6v_0582054c-2090-4cc6-9420-50011ca2d94f/istio-proxy/1.log"
Jul 30 20:28:33.209245 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:33.209219    2554 scope.go:117] "RemoveContainer" containerID="e8229d0979cc4758bda77355ad3192076de3bd0f0ea85bb8a1c60f07fec656b7"
Jul 30 20:28:33.218146 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:33.218131    2554 scope.go:117] "RemoveContainer" containerID="40c9eb1da3fb2c0ba5c2272439eb61e3ffc8e70d45fffab6708463e145760244"
Jul 30 20:28:33.580216 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:33.580183    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54bd4f7fb-wwb4d_749026e1-d4ac-408c-b989-e79e0e95306a/router/0.log"
Jul 30 20:28:33.924114 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:33.924039    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-bzr2x_c3a5e6d2-c06f-4304-9910-fbf0ed329d09/main/0.log"
Jul 30 20:28:34.150640 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:34.150614    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-4j95c_0b90b185-1f26-4137-bd15-017cc5947428/main/0.log"
Jul 30 20:28:39.893278 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:39.893229    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:28:42.521290 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:42.521252    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:28:42.717093 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:42.717060    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-cjhlf_6c766173-91ba-4e51-923b-51ec626a0bee/global-pull-secret-syncer/0.log"
Jul 30 20:28:42.765731 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:42.765700    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log"
Jul 30 20:28:42.896731 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:42.896650    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-180.ec2.internal_ae111a0543c1e08f3cabab9cde34d294/haproxy/0.log"
Jul 30 20:28:47.212499 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:47.212468    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx_2a84b32c-a92e-45e6-ae5c-6a59c1618028/extract/0.log"
Jul 30 20:28:47.239947 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:47.239921    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx_2a84b32c-a92e-45e6-ae5c-6a59c1618028/util/0.log"
Jul 30 20:28:47.259643 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:47.259616    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tq6kx_2a84b32c-a92e-45e6-ae5c-6a59c1618028/pull/0.log"
Jul 30 20:28:47.284430 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:47.284402    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr_32b10063-97db-47e5-bd85-d3dd6633e9f5/extract/0.log"
Jul 30 20:28:47.304380 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:47.304359    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr_32b10063-97db-47e5-bd85-d3dd6633e9f5/util/0.log"
Jul 30 20:28:47.326159 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:47.326134    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08hbfr_32b10063-97db-47e5-bd85-d3dd6633e9f5/pull/0.log"
Jul 30 20:28:47.355058 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:47.355028    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4_e0266cc8-8703-4da9-9029-2170fdf22e9d/extract/0.log"
Jul 30 20:28:47.376337 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:47.376318    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4_e0266cc8-8703-4da9-9029-2170fdf22e9d/util/0.log"
Jul 30 20:28:47.397764 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:47.397746    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738vxg4_e0266cc8-8703-4da9-9029-2170fdf22e9d/pull/0.log"
Jul 30 20:28:47.427057 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:47.427039    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq_29bcabf6-73c0-47a5-8d47-0a92f7a945f7/extract/0.log"
Jul 30 20:28:47.447446 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:47.447424    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq_29bcabf6-73c0-47a5-8d47-0a92f7a945f7/util/0.log"
Jul 30 20:28:47.467615 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:47.467551    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15zfvq_29bcabf6-73c0-47a5-8d47-0a92f7a945f7/pull/0.log"
Jul 30 20:28:48.109586 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:48.109554    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-d9nzc_ace7990c-649f-496b-8cd1-52ca9a626f9f/manager/0.log"
Jul 30 20:28:48.139460 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:48.139439    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-sv8wn_502f8628-f1d0-4b2d-908a-8b09e2211e59/kuadrant-console-plugin/0.log"
Jul 30 20:28:48.198785 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:48.198756    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-rmxcj_c2145a09-d30d-4832-9b2d-40f1beebb16c/registry-server/0.log"
Jul 30 20:28:48.274463 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:48.274425    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-w5ffc_e7accbac-7008-46b2-b1c1-37a50151e649/manager/0.log"
Jul 30 20:28:49.797012 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:49.796981    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5d47a719-7ccd-4b74-9012-201963f6a7ff/alertmanager/0.log"
Jul 30 20:28:49.821605 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:49.821580    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5d47a719-7ccd-4b74-9012-201963f6a7ff/config-reloader/0.log"
Jul 30 20:28:49.851353 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:49.851329    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5d47a719-7ccd-4b74-9012-201963f6a7ff/kube-rbac-proxy-web/0.log"
Jul 30 20:28:49.874472 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:49.874445    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5d47a719-7ccd-4b74-9012-201963f6a7ff/kube-rbac-proxy/0.log"
Jul 30 20:28:49.901890 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:49.901862    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5d47a719-7ccd-4b74-9012-201963f6a7ff/kube-rbac-proxy-metric/0.log"
Jul 30 20:28:49.928589 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:49.928563    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5d47a719-7ccd-4b74-9012-201963f6a7ff/prom-label-proxy/0.log"
Jul 30 20:28:49.959993 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:49.959945    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5d47a719-7ccd-4b74-9012-201963f6a7ff/init-config-reloader/0.log"
Jul 30 20:28:50.000022 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:49.999994    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6f55c4f58d-flnc7_53668666-0a1b-402a-9c46-dd0835d55256/cluster-monitoring-operator/0.log"
Jul 30 20:28:50.192591 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:50.192566    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-59764589d-r26vv_66e316b4-ad9c-43e6-a692-d675438f8af4/metrics-server/0.log"
Jul 30 20:28:50.604142 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:50.604110    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xlpnn_1de08b12-083f-413c-aca5-e4868ed0b718/node-exporter/0.log"
Jul 30 20:28:50.634920 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:50.634891    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xlpnn_1de08b12-083f-413c-aca5-e4868ed0b718/kube-rbac-proxy/0.log"
Jul 30 20:28:50.663451 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:50.663427    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xlpnn_1de08b12-083f-413c-aca5-e4868ed0b718/init-textfile/0.log"
Jul 30 20:28:52.481274 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.481238    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-f77f48c6d-ft7tw_efcb8db8-2128-45ea-9098-ddf248716241/networking-console-plugin/0.log"
Jul 30 20:28:52.829609 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.829524    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fmzs6/perf-node-gather-daemonset-2mfb7"]
Jul 30 20:28:52.829933 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.829918    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c9ed2ee-aae2-41dc-b31b-3207dab9e52e" containerName="istio-proxy"
Jul 30 20:28:52.829995 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.829935    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c9ed2ee-aae2-41dc-b31b-3207dab9e52e" containerName="istio-proxy"
Jul 30 20:28:52.829995 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.829948    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c9ed2ee-aae2-41dc-b31b-3207dab9e52e" containerName="istio-proxy"
Jul 30 20:28:52.829995 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.829970    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c9ed2ee-aae2-41dc-b31b-3207dab9e52e" containerName="istio-proxy"
Jul 30 20:28:52.830090 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.829994    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12138b79-34b5-4bec-96dd-4204d00ee378" containerName="revoke-keys"
Jul 30 20:28:52.830090 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.830003    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="12138b79-34b5-4bec-96dd-4204d00ee378" containerName="revoke-keys"
Jul 30 20:28:52.830090 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.830068    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="12138b79-34b5-4bec-96dd-4204d00ee378" containerName="revoke-keys"
Jul 30 20:28:52.830090 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.830080    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c9ed2ee-aae2-41dc-b31b-3207dab9e52e" containerName="istio-proxy"
Jul 30 20:28:52.830090 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.830086    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c9ed2ee-aae2-41dc-b31b-3207dab9e52e" containerName="istio-proxy"
Jul 30 20:28:52.833164 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.833143    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-2mfb7"
Jul 30 20:28:52.835852 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.835833    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fmzs6\"/\"default-dockercfg-xc2gz\""
Jul 30 20:28:52.835852 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.835843    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fmzs6\"/\"openshift-service-ca.crt\""
Jul 30 20:28:52.837246 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.837087    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fmzs6\"/\"kube-root-ca.crt\""
Jul 30 20:28:52.840586 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.840555    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fmzs6/perf-node-gather-daemonset-2mfb7"]
Jul 30 20:28:52.881025 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.879041    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cab684aa-b5bf-4da1-b708-40cd03b57663-lib-modules\") pod \"perf-node-gather-daemonset-2mfb7\" (UID: \"cab684aa-b5bf-4da1-b708-40cd03b57663\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-2mfb7"
Jul 30 20:28:52.881025 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.879104    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cab684aa-b5bf-4da1-b708-40cd03b57663-podres\") pod \"perf-node-gather-daemonset-2mfb7\" (UID: \"cab684aa-b5bf-4da1-b708-40cd03b57663\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-2mfb7"
Jul 30 20:28:52.881025 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.879141    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cab684aa-b5bf-4da1-b708-40cd03b57663-sys\") pod \"perf-node-gather-daemonset-2mfb7\" (UID: \"cab684aa-b5bf-4da1-b708-40cd03b57663\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-2mfb7"
Jul 30 20:28:52.881025 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.879286    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cab684aa-b5bf-4da1-b708-40cd03b57663-proc\") pod \"perf-node-gather-daemonset-2mfb7\" (UID: \"cab684aa-b5bf-4da1-b708-40cd03b57663\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-2mfb7"
Jul 30 20:28:52.881025 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.879339    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5w7xq\" (UniqueName: \"kubernetes.io/projected/cab684aa-b5bf-4da1-b708-40cd03b57663-kube-api-access-5w7xq\") pod \"perf-node-gather-daemonset-2mfb7\" (UID: \"cab684aa-b5bf-4da1-b708-40cd03b57663\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-2mfb7"
Jul 30 20:28:52.980515 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.980476    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cab684aa-b5bf-4da1-b708-40cd03b57663-lib-modules\") pod \"perf-node-gather-daemonset-2mfb7\" (UID: \"cab684aa-b5bf-4da1-b708-40cd03b57663\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-2mfb7"
Jul 30 20:28:52.980515 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.980519    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cab684aa-b5bf-4da1-b708-40cd03b57663-podres\") pod \"perf-node-gather-daemonset-2mfb7\" (UID: \"cab684aa-b5bf-4da1-b708-40cd03b57663\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-2mfb7"
Jul 30 20:28:52.980776 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.980537    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cab684aa-b5bf-4da1-b708-40cd03b57663-sys\") pod \"perf-node-gather-daemonset-2mfb7\" (UID: \"cab684aa-b5bf-4da1-b708-40cd03b57663\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-2mfb7"
Jul 30 20:28:52.980776 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.980602    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cab684aa-b5bf-4da1-b708-40cd03b57663-proc\") pod \"perf-node-gather-daemonset-2mfb7\" (UID: \"cab684aa-b5bf-4da1-b708-40cd03b57663\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-2mfb7"
Jul 30 20:28:52.980776 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.980630    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w7xq\" (UniqueName: \"kubernetes.io/projected/cab684aa-b5bf-4da1-b708-40cd03b57663-kube-api-access-5w7xq\") pod \"perf-node-gather-daemonset-2mfb7\" (UID: \"cab684aa-b5bf-4da1-b708-40cd03b57663\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-2mfb7"
Jul 30 20:28:52.980776 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.980685    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cab684aa-b5bf-4da1-b708-40cd03b57663-sys\") pod \"perf-node-gather-daemonset-2mfb7\" (UID: \"cab684aa-b5bf-4da1-b708-40cd03b57663\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-2mfb7"
Jul 30 20:28:52.980776 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.980697    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cab684aa-b5bf-4da1-b708-40cd03b57663-proc\") pod \"perf-node-gather-daemonset-2mfb7\" (UID: \"cab684aa-b5bf-4da1-b708-40cd03b57663\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-2mfb7"
Jul 30 20:28:52.980776 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.980684    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cab684aa-b5bf-4da1-b708-40cd03b57663-podres\") pod \"perf-node-gather-daemonset-2mfb7\" (UID: \"cab684aa-b5bf-4da1-b708-40cd03b57663\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-2mfb7"
Jul 30 20:28:52.980776 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.980683    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cab684aa-b5bf-4da1-b708-40cd03b57663-lib-modules\") pod \"perf-node-gather-daemonset-2mfb7\" (UID: \"cab684aa-b5bf-4da1-b708-40cd03b57663\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-2mfb7"
Jul 30 20:28:52.988367 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:52.988342    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5w7xq\" (UniqueName: \"kubernetes.io/projected/cab684aa-b5bf-4da1-b708-40cd03b57663-kube-api-access-5w7xq\") pod \"perf-node-gather-daemonset-2mfb7\" (UID: \"cab684aa-b5bf-4da1-b708-40cd03b57663\") " pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-2mfb7"
Jul 30 20:28:53.106114 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:53.106027    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/1.log"
Jul 30 20:28:53.110466 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:53.110443    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-wrp7d_a6963706-5570-4c07-a0da-707e5bc655a0/console-operator/2.log"
Jul 30 20:28:53.143978 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:53.143927    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-2mfb7"
Jul 30 20:28:53.292894 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:53.292862    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fmzs6/perf-node-gather-daemonset-2mfb7"]
Jul 30 20:28:53.294638 ip-10-0-134-180 kubenswrapper[2554]: W0730 20:28:53.294610    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcab684aa_b5bf_4da1_b708_40cd03b57663.slice/crio-39b25736be0c3a14d386355da3674f9f835bcc385c9525fd1daa51b0ad26fc07 WatchSource:0}: Error finding container 39b25736be0c3a14d386355da3674f9f835bcc385c9525fd1daa51b0ad26fc07: Status 404 returned error can't find the container with id 39b25736be0c3a14d386355da3674f9f835bcc385c9525fd1daa51b0ad26fc07
Jul 30 20:28:53.616147 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:53.616117    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5f8954cf7f-gnzjs_8a9c2ffe-2cb6-4381-b125-6b2e7cb26a10/console/0.log"
Jul 30 20:28:54.201737 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:54.201692    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-2mfb7" event={"ID":"cab684aa-b5bf-4da1-b708-40cd03b57663","Type":"ContainerStarted","Data":"0130f529ad2abc47b565e35c5d24877a4f6434a39c6dbcb86aefb163c2337bde"}
Jul 30 20:28:54.201924 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:54.201834    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-2mfb7"
Jul 30 20:28:54.201924 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:54.201864    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-2mfb7" event={"ID":"cab684aa-b5bf-4da1-b708-40cd03b57663","Type":"ContainerStarted","Data":"39b25736be0c3a14d386355da3674f9f835bcc385c9525fd1daa51b0ad26fc07"}
Jul 30 20:28:54.204504 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:54.204481    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5b8776669-hlccf_82b54e54-02dd-4fb7-81d2-97b76aa991fe/volume-data-source-validator/0.log"
Jul 30 20:28:54.218594 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:54.218532    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-2mfb7" podStartSLOduration=2.2185190280000002 podStartE2EDuration="2.218519028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 20:28:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 20:28:54.217376907 +0000 UTC m=+3862.181957989" watchObservedRunningTime="2026-07-30 20:28:54.218519028 +0000 UTC m=+3862.183100141"
Jul 30 20:28:55.247542 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:55.247511    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wh4lj_c6db7b86-8d56-4418-be35-098fab4e24da/dns/0.log"
Jul 30 20:28:55.267595 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:55.267572    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wh4lj_c6db7b86-8d56-4418-be35-098fab4e24da/kube-rbac-proxy/0.log"
Jul 30 20:28:55.331488 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:55.331462    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lk9m5_5028ef75-1c24-4d52-8364-8ff8ac66f6cf/dns-node-resolver/0.log"
Jul 30 20:28:55.869500 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:55.869468    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cdmwn_5bdbf9db-4a68-4625-adc1-a17b0e04d581/node-ca/0.log"
Jul 30 20:28:56.863849 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:56.863821    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv_2c97707f-d73f-4d27-84f7-3152b9b709ea/istio-proxy/0.log"
Jul 30 20:28:57.084904 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:57.084875    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-fbm6v_0582054c-2090-4cc6-9420-50011ca2d94f/istio-proxy/0.log"
Jul 30 20:28:57.118977 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:57.118881    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-fbm6v_0582054c-2090-4cc6-9420-50011ca2d94f/istio-proxy/1.log"
Jul 30 20:28:57.199363 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:57.199333    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54bd4f7fb-wwb4d_749026e1-d4ac-408c-b989-e79e0e95306a/router/0.log"
Jul 30 20:28:58.015207 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:58.015181    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hqnjj_cc17657c-96a1-4e8f-b659-dd8a92b29ee0/serve-healthcheck-canary/0.log"
Jul 30 20:28:58.459178 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:58.459153    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-smtsw_471ab30c-b470-404b-b10a-d72642865017/insights-operator/0.log"
Jul 30 20:28:58.460146 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:58.460130    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-smtsw_471ab30c-b470-404b-b10a-d72642865017/insights-operator/1.log"
Jul 30 20:28:58.539369 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:58.539340    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6ddl4_9584099b-badd-46a7-b27d-c62012611e0a/kube-rbac-proxy/0.log"
Jul 30 20:28:58.557873 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:58.557851    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6ddl4_9584099b-badd-46a7-b27d-c62012611e0a/exporter/0.log"
Jul 30 20:28:58.577881 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:28:58.577860    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6ddl4_9584099b-badd-46a7-b27d-c62012611e0a/extractor/0.log"
Jul 30 20:29:00.219608 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:29:00.219584    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fmzs6/perf-node-gather-daemonset-2mfb7"
Jul 30 20:29:01.033947 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:29:01.033913    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-785db54bd5-c59kr_f9e40da3-e49e-4085-8d78-749bc2f9394c/manager/0.log"
Jul 30 20:29:01.218843 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:29:01.218814    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr_a1391f7f-8cee-46c8-940f-167d78cd70e3/manager/0.log"
Jul 30 20:29:01.248554 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:29:01.248524    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76fdfbdbf4-j4fzr_a1391f7f-8cee-46c8-940f-167d78cd70e3/copy-manifests/0.log"
Jul 30 20:29:02.765261 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:29:02.765233    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-659fd749fb-tnqm6_8e322bc6-457c-418a-abd4-63acc15d5cca/manager/0.log"
Jul 30 20:29:02.963311 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:29:02.963284    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv_2c97707f-d73f-4d27-84f7-3152b9b709ea/istio-proxy/0.log"
Jul 30 20:29:03.162515 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:29:03.162475    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-fbm6v_0582054c-2090-4cc6-9420-50011ca2d94f/istio-proxy/0.log"
Jul 30 20:29:03.189426 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:29:03.189392    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-fbm6v_0582054c-2090-4cc6-9420-50011ca2d94f/istio-proxy/1.log"
Jul 30 20:29:03.238962 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:29:03.238875    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54bd4f7fb-wwb4d_749026e1-d4ac-408c-b989-e79e0e95306a/router/0.log"
Jul 30 20:29:03.640720 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:29:03.640643    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrw8v_c7d624a3-3eef-42fe-b07a-f5518e4b1c32/konnectivity-agent/0.log" limit=1024
Jul 30 20:29:04.604152 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:29:04.604121    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-qclpv_2c97707f-d73f-4d27-84f7-3152b9b709ea/istio-proxy/0.log"
Jul 30 20:29:04.804586 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:29:04.804548    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-fbm6v_0582054c-2090-4cc6-9420-50011ca2d94f/istio-proxy/0.log"
Jul 30 20:29:04.833226 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:29:04.833189    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-fbm6v_0582054c-2090-4cc6-9420-50011ca2d94f/istio-proxy/1.log"
Jul 30 20:29:04.893344 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:29:04.893306    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54bd4f7fb-wwb4d_749026e1-d4ac-408c-b989-e79e0e95306a/router/0.log"
Jul 30 20:29:07.908063 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:29:07.908025    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-72ngp_083469f7-657e-4dc8-98ac-70c9749d5922/kube-storage-version-migrator-operator/1.log"
Jul 30 20:29:07.908915 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:29:07.908892    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-72ngp_083469f7-657e-4dc8-98ac-70c9749d5922/kube-storage-version-migrator-operator/0.log"
Jul 30 20:29:08.857323 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:29:08.857297    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5j6j7_3ad1522f-8aea-4551-af0c-131bf223b514/kube-multus-additional-cni-plugins/0.log"
Jul 30 20:29:08.893078 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:29:08.893052    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5j6j7_3ad1522f-8aea-4551-af0c-131bf223b514/egress-router-binary-copy/0.log"
Jul 30 20:29:08.924230 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:29:08.924205    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5j6j7_3ad1522f-8aea-4551-af0c-131bf223b514/cni-plugins/0.log"
Jul 30 20:29:08.965006 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:29:08.964978    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5j6j7_3ad1522f-8aea-4551-af0c-131bf223b514/bond-cni-plugin/0.log"
Jul 30 20:29:09.001006 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:29:09.000974    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5j6j7_3ad1522f-8aea-4551-af0c-131bf223b514/routeoverride-cni/0.log"
Jul 30 20:29:09.033244 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:29:09.033166    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5j6j7_3ad1522f-8aea-4551-af0c-131bf223b514/whereabouts-cni-bincopy/0.log"
Jul 30 20:29:09.069099 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:29:09.069070    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5j6j7_3ad1522f-8aea-4551-af0c-131bf223b514/whereabouts-cni/0.log"
Jul 30 20:29:09.471211 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:29:09.471185    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-pdw24_6db8c394-338e-4a79-a776-1750a359ffa9/kube-multus/0.log"
Jul 30 20:29:09.563475 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:29:09.563444    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dtpk6_aec35b15-c096-4e99-a7dc-77be12a9ceca/network-metrics-daemon/0.log"
Jul 30 20:29:09.582225 ip-10-0-134-180 kubenswrapper[2554]: I0730 20:29:09.582197    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dtpk6_aec35b15-c096-4e99-a7dc-77be12a9ceca/kube-rbac-proxy/0.log"